Generalized cross correlation (GCC) is arguably the most widely used time difference of arrival (TDOA) estimators. Most studies on GCC assume spatially uncorrelated noises, and focus on the design of weighting functions. However, real world noises tend to be spatially correlated, and such correlation introduces phase bias to the cross phase spectra used in GCC. This paper proposes a simple closed-form maximum likelihood (ML) cross phase spectra estimator for GCC by only assuming the prior knowledge of noise spatial complex coherence function, which can be predicted in theory or measured in advance. TDOA estimation experimental results on real world speech recordings confirm the usefulness of this simple new phase estimator.

We study a fundamental generalization of principal component analysis (PCA) that looks for a small set of common time series, i.e., principal components (PCs), whose filtered versions can explain most of the variances of multiple observed time series. This problem boils down to PCA in the frequency domain, in principle. But, frequency domain processing suffers from aliasing, and brings inconveniences when handling certain time domain properties like nonstationarity, and sparsity. We propose novel time, and z-domain costs for such PCA, and study its properties in detail with setting of either finite or diverging filter lengths. We further discuss its implementations, and possible extensions, and present numerical results for empirical performance study. Convolution is used to extract these PCs, thus the name convolutional PCA.

This letter studies nearly perfect reconstruction periodic sequences modulated filter banks, which includes many useful examples, e.g., the discrete Fourier transform (DFT), generalized DFT, and cosine-modulated ones. A general framework is developed to meet arbitrary, but feasible design constraints, e.g., length of filters, system delay, decimation ratio, phase linearity, etc., and it also reveals new design freedoms, i.e., phase shifts in the modulation sequences. An efficient Newton algorithm is proposed, and a MATLAB/Octave package with diverse design examples is provided. Numerical results show that better designs can be obtained by searching for the solutions in a larger space than the one from traditional frameworks.

Stochastic gradient descent (SGD) still is the workhorse for many practical problems. However, it converges slow, and can be difficult to tune. It is possible to precondition SGD to accelerate its convergence remarkably. But many attempts in this direction either aim at solving specialized problems, or result in significantly more complicated methods than SGD. This paper proposes a new method to adaptively estimate a preconditioner, such that the amplitudes of perturbations of preconditioned stochastic gradient match that of the perturbations of parameters to be optimized in a way comparable to Newton method for deterministic optimization. Unlike the preconditioners based on secant equation fitting as done in deterministic quasi-Newton methods, which assume positive definite Hessian and approximate its inverse, the new preconditioner works equally well for both convex and nonconvex optimizations with exact or noisy gradients. When stochastic gradient is used, it can naturally damp the gradient noise to stabilize SGD. Efficient preconditioner estimation methods are developed, and with reasonable simplifications, they are applicable to large-scale problems. Experimental results demonstrate that equipped with the new preconditioner, without any tuning effort, preconditioned SGD can efficiently solve many challenging problems like the training of a deep neural network or a recurrent neural network requiring extremely long-term memories.

Estimating likelihood or entropy rate is one of the key issues in many signal processing problems. Mutual information rate, which leads to the minimization of entropy rate, provides a natural cost for achieving blind source separation (BSS). In many complex-valued BSS applications, the latent sources are non-Gaussian, noncircular, and possess sample dependence. Consequently, an effective estimator of entropy rate that jointly considers all three properities of the sources is required. In this paper, we propose such an entropy rate estimator that assumes the sources are generated by invertible filters. With this new entropy rate estimator, we propose a complex entropy rate bound minimization algorithm. Simulation results show that the new method exploits all three properties effectively.

Since in many blind source separation applications, latent sources are both non-Gaussian and have sample dependence, it is desirable to exploit both non-Gaussianity and sample dependency. In this paper, we use the Markov model to construct a general framework for the analysis and derivation of algorithms that take both properties into account. We also present two algorithms using two effective source priors. The first one is a multivariate generalized Gaussian distribution and the second is an autoregressive model driven by a generalized Gaussian distributed process. We derive the Cramér-Rao lower bound and demonstrate that the performance of the algorithms approach the lower bound especially when the underlying model matches the parametric model. We also demonstrate that a flexible semi-parametric approach exhibits very desirable performance.

Matrix optimization of cost functions is a common problem. Construction of methods that enable each row or column to be individually optimized, i.e., decoupled, are desirable for a number of reasons. With proper decoupling, the convergence characteristics such as local stability can be improved. Decoupling can enable density matching in applications such as independent component analysis (ICA). Lastly, efficient Newton algorithms become tractable after decoupling. The most common method for decoupling rows is to reduce the optimization space to orthogonal matrices. Such restrictions can degrade performance. We present a decoupling procedure that uses standard vector optimization procedures while still admitting nonorthogonal solutions. We utilize the decoupling procedure to develop a new decoupled ICA algorithm that uses Newton optimization enabling superior performance when the sample size is limited.

A decoupled relative Newton algorithm is proposed for the matrix optimization problem encountered in blind source separation (BSS) and independent component analysis (ICA). The algorithm decouples the matrix optimization problem into a series of small vector optimization problems. The nonsingularity of separation matrix enables a simple and efficient relative Newton learning algorithm for the vector optimization problems. Simulation results are reported to confirm its superior performance.

This paper considers the problem of joint blind source separation (J-BSS), which appears in many practical problems such as blind deconvolution or functional magnetic resonance imaging (fMRI). In particular, we establish the necessary and sufficient conditions for the solution of the J-BSS problem by exclusively exploiting the second-order statistics (SOS) of the observations. The identifiability analysis is based on the idea of equivalently distributed sets of latent variables, that is, latent variables with covariance matrices related by means of a diagonal matrix. Interestingly, the identifiability analysis also allows us to introduce a measure of the identifiability degree based on Kullback-Leibler projections. This measure is clearly correlated with the performance of practical SOS-based J-BSS algorithms, which is illustrated by means of numerical examples.

In this paper, the blind and semiblind channel estimation and equalization are investigated for zero-padding single-carrier block transmission (ZP-SCBT) systems using second-order statistics. Unlike the conventional channel estimation techniques, the proposed approach focuses on identifying the inverse of the channel impulse response rather than the channel response itself by skillfully exploiting the redundancy of the zero-padding. One interesting advantage of the proposed blind and semiblind approach is that the number of received blocks needed for blind identification is significantly reduced compared to the subspace method. Moreover, the computational complexity of the proposed approaches is much lower than the existing subspace methods. Simulation results are provided to demonstrate that the performance of the proposed approach is superior to the subspace-based blind methods.

It has been shown that using minimum error entropy as the cost function leads to important performance gains in adaptive filtering, especially when the Gaussianity assumptions on the error distribution do not hold. In this paper, we show that by using the entropy bound rather than the entropy, we can derive an efficient algorithm for supervised training. We demonstrate its effectiveness by a system identification problem using a generalized Gaussian noise model.

We propose a new entropy rate estimator for a second and/or higher-order correlated source by modeling it as the output of a linear filter, which can be mixed-phase, driven by Gaussian or non-Gaussian noise. Based on this estimator, we develop a new spatiotemporal blind source separation (BSS) algorithm, full BSS (FBSS), by minimizing the entropy rate of separated sources. FBSS provides more flexibilities in exploiting the temporal structures of sources than the state-of-the-art spatiotemporal BSS algorithms, which use AR models, and thus imply minimum-phase property of sources.

Independent component analysis (ICA) has been successfully applied for the analysis of functional magnetic resonance imaging (fMRI) data. However, independence might be too strong a constraint for certain sources. In this paper, we present an independent subspace analysis (ISA) framework that forms independent subspaces among the estimated sources having dependencies by a hierarchial clustering approach and subsequently separates the dependent sources in the task-related subspace using prior information. We study the incorporation of two types of prior information to transform the sources within the task-related subspace: sparsity and task-related time courses. We demonstrate the effectiveness of our proposed method for source separation of multi-subject fMRI data from a visuomotor task. Our results show that physiologically meaningful dependencies among sources can be identified using our subspace approach and the dependent estimated components can be further separated effectively using a subsequent transformation.

A low complexity algorithm is proposed for estimating the multipath time-of-arrival (TOA) and attenuation factors from a noisy received signal consisting of multiple overlapped echoes. Different from the conventional projection onto convex sets (POCS) method, the proposed approach harnesses the sparse property of multipath channel and the ℓ1-norm is adopted as the measurement of sparsity. The TOA estimation problem is solved by a series of projection onto convex sets, including hypersphere and hyper-polyhedra. The computational complexity of the modified POCS algorithm is O(N log N) per iteration with N being the length of the received signal. Simulation results confirm that the proposed approach provides better performance in terms of temporal resolution and robustness to noise compared with the matched filtering and the conventional POCS method.

Two Newton-type algorithms using the generalized complex modulus (GCM) criterion for blind equalization and carrier phase recovery are proposed. First the partial Hessian and full Hessian of the real GCM loss function with complex valued arguments are calculated by second-order differential. Then an adaptive pseudo Newton learning algorithm and a full Newton learning algorithm are designed. By using the matrix inversion lemma, both Newton algorithms can be implemented with a computational complexity of O(L2) efficiently, where L is the length of equalizer. Simulation results demonstrate that the two Newton algorithms can achieve automatic carrier phase recovery and exhibit fast convergence rates.

A new signal-selective direction finding algorithm which exploits the property of the cyclostationarity of incoming signals is proposed. After dimensionality reducing by projecting the observed array data onto the signal subspace, the array manifold matrix is identified by the simultaneous diagonalization structure of the matrix pencil consisting of the cyclic correlation matrix and the cyclic conjugate correlation matrix. Then the direction-of-arrivals (DOAs) are obtained from the phase-differences of the estimated array manifold matrix. Simulation results demonstrate that the proposed algorithm is superior to the cyclic-MUSIC and cyclic-ESPRIT in terms of the root mean squares errors (RMSEs) of the DOA estimates.

A new algorithm for direction-of-arrival (DOA) estimation of non-Gaussian sources is proposed. Based on the joint diagonalization structure of multiple fourth-order cumulant matrices, a novel cost function is designed and a new spatial spectrum for direction finding at hand is derived. Unlike the subspace-based techniques, it is not necessary to determine the number of sources in advance for the proposed algorithm. Moreover, the proposed method is insensitive to the spatially correlated noise. Simulation results are provided to demonstrate the performance of the proposed approach.

A new algorithm for estimating the frequencies of the two- dimensional (2-D) exponentials is presented. Via joint diagonalization of a set of matrices, the estimation of the automatically paired 2-D frequency parameters are obtained. Hence the proposed algorithm can avoid the computationally expensive 2-D parameters pairing procedure. Simulation results validate the high performance of the new algorithm.

Based on an energy conservation relation, the tracking performance of a family of generalized constant modulus algorithm (GCMA) is analyzed in this paper. Both theoretical analysis and simulation results validate the excellent tracking performance of the extended constant modulus algorithm (ECMA) in some situations.

This letter proposes a novel second-order statistics-based sequential blind extraction algorithm via approximate joint diagonalization of autocorrelation matrices. The proposed algorithm is computationally simple and efficient and can extract specific sources if some a priori knowledge is available. An experiment on real-world ECG data validates the good performance of the proposed algorithm

In this letter, the super-exponential algorithm (SEA) is interpreted as a gradient search of an extremum of the inverse filter criteria, and the optimal gradient search step-size is presented as the root of a quartic equation. Different from the gradient search interpretation presented by Mboup and Regalia in the combined channel-equalizer space, the interpretation here is directly related with the realization of SEA in the equalizer space and uncovers more convergence details of SEA under unideal conditions.

By generalizing the definition of complex modulus, this letter presents a family of generalized constant modulus algorithms whose special examples includes not only the well-known constant modulus algorithm, but also the recently proposed sign Godard algorithm, square contour algorithm (SCA), generalized SCA, and sign SCA