Independent deeply learned matrix analysis (IDLMA) is a state-of-the-art determined audio source separation method based on pretrained deep neural networks (DNNs). Owing to the excellent expression power of DNNs, IDLMA can handle a wider range of sources than conventional source models such as nonegative matrix factorization (NMF). However, owing to its supervised nature, the separation performance of IDLMA often degrades in the presence of timbral mismatches between the training data and the to-be-separated data. In this paper, we propose two source models that encompass the NMF- and DNN-based source models by constructing a prior distribution of the source power spectrogram (product of priors: PoP) on the basis of the product-of-expert concept. Since the NMF-based source model works well for a fully blind situation, the proposed models can handle the timbral mismatch without losing the expression power of DNNs. By introducing the PoP-based source models into IDLMA, we propose IDLMA extensions (PoP-IDLMAs) and derive their efficient parameter estimation algorithms on the basis of the majorization–minimization algorithm. Experimental results demonstrated the effectiveness of the proposed PoP-IDLMAs and that the proposed models greatly improve the source power estimation in frequency bands above 500 Hz.

A differentiable digital signal processing (DDSP) autoencoder is a musical sound synthesizer that combines a deep neural network (DNN) and spectral modeling synthesis. It allows us to flexibly edit sounds by changing the fundamental frequency, timbre feature, and loudness (synthesis parameters) extracted from an input sound. However, it is designed for a monophonic harmonic sound and cannot handle mixtures of harmonic sounds. In this paper, we propose a model (DDSP mixture model) that represents a mixture as the sum of the outputs of multiple pretrained DDSP autoencoders. By fitting the output of the proposed model to the observed mixture, we can directly estimate the synthesis parameters of each source. Through synthesis parameter extraction experiments, we show that the proposed method has high and stable performance compared with a straightforward method that applies the DDSP autoencoder to the signals separated by an audio source separation method.

Multichannel nonnegative matrix factorization (MNMF) is a common blind source separation technique that employs full-rank spatial covariance matrices (SCMs). The full-rank SCMs can simulate reverberant mixing systems where the sources are spatially spread. In conventional MNMF, spectrograms of observed signals are modeled by some types of distribution, e.g., the Gaussian distribution and Student’s t distribution. However, MNMF based on the sub-Gaussian distribution has not been proposed because its cost function is difficult to minimize. In this paper, we address the statistical model extension of MNMF to the sub-Gaussian distribution to improve the source separation accuracy. In the proposed method, the generalized Gaussian distribution is utilized as the sub-Gaussian model. Moreover, to design an auxiliary function for the proposed cost function, we introduce the joint-diagonalizability constraint to SCMs similarly to FastMNMF. Two types of update rule for the proposed MNMF are derived on the basis of the majorization-minimization (MM) and majorization-equalization (ME) algorithms. Since the optimization speed of each parameter affects the source separation performance, we experimentally analyze the best combination of MM- and ME-algorithm-based update rules in the proposed method. Experiments of blind source separation reveal that the proposed MNMF based on the sub-Gaussian model can outperform conventional methods.

Independent deeply learned matrix analysis (IDLMA) is a fast and high-performance method for multichannel audio source separation. IDLMA utilizes the deep neural network inference of source models and the blind estimation of demixing filters based on source independence. In conventional IDLMA, iterative projection (IP) is exploited to estimate the demixing filters. Although IP is a fast algorithm, it sometimes fails to estimate an appropriate solution. This is because IP updates the demixing filters in a sourcewise manner, where only one source model is used for each update, and the update sometimes becomes unstable owing to the specific low-quality source models. In this paper, we first derive a new numerically stable microphone-wise update algorithm that exploits all source model information simultaneously. The microphone-wise update problem cannot be solved by IP; instead, a new type of vectorwise coordinate descent algorithm is introduced. Next, comparison analysis of the proposed microphone-wise update and IP reveals the tradeoff w.r.t. convergence speed and numerical stability. To resolve this tradeoff problem, we propose the automatic selection of update rules on the basis of the likelihood function of observed signals. Finally, experimental results show the efficacy of the proposed IDLMA with the automatic selection of update rules.

In this paper, we address a convolutive blind source separation (BSS) problem and propose a new extended framework of FastMNMF by introducing prior information for joint diagonalization of the spatial covariance matrix model. Recently, FastMNMF has been proposed as a fast version of multichannel nonnegative matrix factorization under the assumption that the spatial covariance matrices of multiple sources can be jointly diagonalized. However, its source-separation performance was not improved and the physical meaning of the joint-diagonalization process was unclear. To resolve these problems, we first reveal a close relationship between the joint-diagonalization process and the demixing system used in independent low-rank matrix analysis (ILRMA). Next, motivated by this fact, we propose a new regularized FastMNMF supported by ILRMA and derive convergence-guaranteed parameter update rules. From BSS experiments, we show that the proposed method outperforms the conventional FastMNMF in source-separation accuracy with almost the same computation time.

Independent low-rank matrix analysis (ILRMA) is a fast and stable method of blind audio source separation. Conventional ILRMAs assume time-variant (super-)Gaussian source models, which can only represent signals that follow a super-Gaussian distribution. In this article, we focus on ILRMA based on a generalized Gaussian distribution (GGD-ILRMA) and propose a new type of GGD-ILRMA that adopts a time-variant sub-Gaussian distribution for the source model. We propose a new update scheme called generalized iterative projection for homogeneous source models (GIP-HSM) and obtain a convergence-guaranteed update rule for demixing spatial parameters by combining the GIP-HSM scheme and the majorization-minimization (MM) algorithm. Furthermore, a new extension of the MM algorithm is proposed for the convergence acceleration by applying the majorization-equalization algorithm to a multivariate case. In the experimental evaluation, we show the versatility of the proposed method, i.e., the proposed time-variant sub-Gaussian source model can be applied to various types of source signal.

Audio source separation is an important problem for many audio applications. Independent low-rank matrix analysis (ILRMA) is a recently proposed algorithm that employs the statistical independence between sources and the low-rankness of the time-frequency structure in each source. As reported in this paper, we have developed a new framework that enables us to introduce a spatial regularization of the demixing matrix in ILRMA. Since the conventional optimization cannot be applied to this regularized ILRMA, we derive a novel approach based on vectorwise coordinate descent, which does not require a step-size parameter and guarantees convergence. In experiments, ILRMA with beamforming-based regularization is evaluated as an application of the proposed framework.

In this paper, to address problems in multichannel music signal separation, we propose a new hybrid method that combines directional clustering and advanced nonnegative matrix factorization (NMF). The aims of multichannel music signal separation technology is to extract a specific target signal from observed multichannel signals that contain multiple instrumental sounds. In previous studies, various methods using NMF have been proposed, but many problems remain including poor separation accuracy and lack of robustness. To solve these problems, we propose a new supervised NMF (SNMF) with spectrogram restoration and a hybrid method that concatenates the proposed SNMF after directional clustering. Via the extrapolation of supervised spectral bases, the proposed SNMF attempts both target signal separation and reconstruction of the lost target components, which are generated by preceding directional clustering. In addition, we experimentally reveal the trade-off between separation and extrapolation abilities and propose a new scheme for adaptive divergence, where the optimal divergence can be automatically changed in each time frame according to the local spatial conditions. The results of an evaluation experiment show that our proposed hybrid method outperforms the conventional music signal separation methods.

In this paper, we propose a new approach for addressing music signal separation based on the generalized Bayesian estimator with automatic prior adaptation. This method consists of three parts, namely, the generalized MMSE-STSA estimator with a flexible target signal prior, the NMF-based dynamic interference spectrogram estimator, and closed-form parameter estimation for the statistical model of the target signal based on higher-order statistics. The statistical model parameter of the hidden target signal can be detected automatically for optimal Bayesian estimation with online target-signal prior adaptation. Our experimental evaluation can show the efficacy of the proposed method.

In this paper, we propose a musical-noise-free blind speech extraction method using a microphone array for application to nonstationary noise. In our previous study, it was found that optimized iterative spectral subtraction (SS) results in speech enhancement with almost no musical noise generation, but this method is valid only for stationary noise. The proposed method consists of iterative blind dynamic noise estimation by, e.g., independent component analysis (ICA) or multichannel Wiener filtering, and musical-noise-free speech extraction by modified iterative SS, where multiple iterative SS is applied to each channel while maintaining the multichannel property reused for the dynamic noise estimators. Also, in relation to the proposed method, we discuss the justification of applying ICA to signals nonlinearly distorted by SS. From objective and subjective evaluations simulating a real-world hands-free speech communication system, we reveal that the proposed method outperforms the conventional methods.

A single-channel dereverberation method with low computational complexity is proposed. We introduce the complementary Wiener filter which can suppress a late reverberation during silence intervals via theoretical analysis and numerical calculation. An implementation represents reductions both of memory consumption and operative calculations compared to a conventional method; each reduction is almost by half. Dereverberation performance is evaluated by an experimental simulation using speech signals and measured room impulse responses. The performance under several hundreds of msec of the reverberation time is similar to the conventional method: 6 [dB] reverberation reduction and 3 [dB] improvement of target-to-interference ratio.

In this paper, we propose a new theory of nonlinear noise reduction with a perfectly musical-noise-free property, where no musical noise is generated even for a high signal-to-noise ratio. To achieve high-quality noise reduction with low musical noise, an iterative spectral subtraction method, i.e., recursively applied weak nonlinear signal processing, has been proposed. Although evaluation experiments indicated the existence of an appropriate parameter setting that gives a musical-noise-free state, no theoretical studies have been carried out. Therefore, in this paper, we theoretically derive pairs of internal parameters that satisfy the musical-noise-free condition by analysis based on higher-order statistics. It is clarified that finding a fixed point in the kurtosis of noise spectra enables the reproduction of the musical-noise-free state, and comparative experiments with commonly used noise reduction methods show the efficacy of the proposed method.

In this paper, we provide a theoretical analysis of the amount of musical noise in iterative spectral subtraction, and its optimization method for the least musical noise generation. To achieve high-quality noise reduction with low musical noise, iterative spectral subtraction, i.e., iteratively applied weak nonlinear signal processing, has been proposed. Although the effectiveness of the method has been reported experimentally, there have been no theoretical studies. Therefore, in this paper, we formulate the generation process of musical noise by tracing the change in kurtosis of noise spectra, and conduct a comparison of the amount of musical noise for different parameter settings but the same achieved level of noise attenuation. Furthermore, we theoretically derive the optimal internal parameters that generate no musical noise. It is clarified that to find a fixed point in kurtosis yields the no-musical-noise property. Comparative experiments with commonly used noise reduction methods show the proposed method's efficacy.

Recently, one of the authors has reported that the amount of generated musical noise is strongly correlated with higher-order statistics of the power spectra. On the basis of this finding, in this paper, we provide a new theoretical analysis of the amount of musical noise generated via the Wiener filtering family. Our theoretical analysis allows the universal performance description from the viewpoint of the amount of musical noise generation and that of noise reduction, enabling reasonable sound quality comparison under the same noise reduction performance. From a mathematical analysis and evaluation experiments, we also clarify which parameter settings result in less musical noise generated in the Wiener filtering family.

In this paper, we provide a new theoretical analysis of the amount of musical noise generated via generalized spectral subtraction based on higher order statistics. Power spectral subtraction is the most commonly used spectral subtraction method, and in our previous study a musical noise assessment theory limited to the power spectral domain was proposed. In this paper, we propose a generalization of our previous theory on spectral subtraction for arbitrary exponent parameters. We can thus compare the amount of musical noise between any exponent domains from the results of our analysis. We also clarify that less musical noise is generated when we choose a lower exponent spectral domain; this implies that there is no theoretical justification for using power/amplitude spectral subtraction.

In this paper, we propose a musical-noise-controllable algorithm for array signal processing with the aim for high-performance and high-quality noise reduction. Recently, many methods of integrating linear microphone array signal processing and nonlinear signal processing for noise reduction have been studied, but these methods often suffer from the problem of musical noise. In the proposed algorithm, channelwise spectral subtraction is applied before adaptive array signal processing. We also introduce a new automatic control algorithm to obtain the subtraction strength parameter used in the spectral subtraction, which depends on the amount of generated musical noise, measured by higher order statistics. We confirm the effectiveness of the proposed algorithm via objective and subjective evaluations.

In this paper, we conduct a theoretical analysis of the amount of musical noise generated via methods of integrating beamforming and spectral subtraction (SS) based on higher-order statistics under the same noise reduction performance condition. In our previous analysis, we did not consider the effect of flooring technique in SS and the fact that the noise reduction performances of the integration methods are not equivalent. Then, in this study, we analyze the amount of generated musical noise with consideration of such problems. As a result of the analysis, it is clarified that an appropriate structure depends on both the parameters of SS and the statistical characteristics of the input signal. Moreover, it is also revealed that a specific structure is proper to reduce the musical noise for almost all cases.

In this paper, we conduct an analysis for reduction of musical noise in integration method of microphone array signal processing and nonlinear signal processing. In these days, for better noise reduction, integration methods of microphone array signal processing and nonlinear signal processing have been researched. However, non-linear signal processing causes musical noise. Since such musical noise make users uncomfortable, it is desired that musical noise is mitigated. Moreover, in these days, it is reported that higher-order statistics is strongly related with the amount of generated musical noise. Thus, we analyze the integrated method of microphone array signal processing and nonlinear signal processing, based on higher-order statistics. Also, we propose an architecture for reducing musical noise based on the analysis. The effectiveness of the proposed architecture and analysis correctness are shown via a computer simulation and a subjective evaluation.

In this paper, we reveal new findings about the generated musical noise in minimum mean-square error short-time spectral amplitude (MMSE STSA) processing. Recently we have proposed a objective metric of musical noise based on kurtosis change ratio on spectral subtraction (SS). Also we found an interesting relationship among the degree of generated musical noise, the shapes of signal-s probability density function, the strength parameter of SS processing. This paper is aimed to automatically evaluate the sound quality of various types of noise reduction methods using kurtosis change ratio. We give a mathematical analysis based on higher-order statistics viewpoint, and lead to a valuable relation in that MMSE STSA has a weakness in speech period distortion rather than noise period, and vice versa in SS.