The computational complexity of classical time domain gradient-based echo cancellation algorithms might be prohibitively high, due to the very long response of the acoustic transfer functions involved. A reduction in computational complexity can be achieved by using frequency domain or subband algorithms. However, these algorithms introduce an inherent delay in the signal path. The delayed echo has an annoying psychoacoustic effect. Additionally, the delay prevents natural, full-duplex conversation. Moreover, when operated in practical scenarios, using speech signals in actual room acoustic environments, the convergence and tracking properties of the frequency domain algorithms do not compare favorably with those of the NLMS algorithm. This is because the range of values of the convergence constant that support a stable filter is more restrictive for the frequency domain algorithms. In this study we introduce a new algorithm termed delayless frequency domain (DLFD). The DLFD exhibits performance comparable to that of the NLMS algorithm with a computational complexity comparable to that of standard frequency domain algorithms and without the processing delay.

Approximate model equations often relate given measurements to unknown parameters whose estimate is sought. The least-squares (LS) estimation criterion assumes the measured data to be exact, and seeks parameters which minimize the model errors. Existing extensions of LS, such as the total LS (TLS) and constrained TLS (CTLS) take the opposite approach, namely assume the model equations to be exact, and attribute all errors to measurement inaccuracies. We introduce the extended LS (XLS) criterion, which accommodates both error sources. We define 'pseudo-linear' models, with which we provide an iterative algorithm for minimization of the XLS criterion. Under certain statistical assumptions, we show that XLS coincides with a statistical criterion, which we term the 'joint maximum-a-posteriori-maximum-likelihood' (JMAP-ML) criterion. We identify the differences between the JMAP-ML and ML criteria, and explain the observed superiority of JMAP-ML over ML under non-asymptotic conditions.

Speech quality and intelligibility might significantly deteriorate in the presence of background noise, especially when the speech signal is subject to subsequent processing. In particular, speech coders and automatic speech recognition (ASR) systems that were designed or trained to act on clean speech signals might be rendered useless in the presence of background noise. Speech enhancement algorithms have therefore attracted a great deal of interest. In this paper, we present a class of Kalman filter-based algorithms with some extensions, modifications, and improvements of previous work. The first algorithm employs the estimate-maximize (EM) method to iteratively estimate the spectral parameters of the speech and noise parameters. The enhanced speech signal is obtained as a byproduct of the parameter estimation algorithm. The second algorithm is a sequential, computationally efficient, gradient descent algorithm. We discuss various topics concerning the practical implementation of these algorithms. Extensive experimental study using real speech and noise signals is provided to compare these algorithms with alternative speech enhancement algorithms, and to compare the performance of the iterative and sequential algorithms.

Speech quality and intelligibility might significantly deteriorate in the presence of background noise, especially when the speech signal is subject to subsequent processing. In this paper we represent a class of Kalman-filter based speech enhancement algorithms with some extensions, modifications, and improvements. The first algorithm employs the estimate-maximize (EM) method to iteratively estimate the spectral parameters of the speech and noise parameters. The enhanced speech signal is obtained as a by-product of the parameter estimation algorithm. The second algorithm is a sequential, computationally efficient, gradient descent algorithm. We discuss various topics concerning the practical implementation of these algorithms. Experimental study, using real speech and noise signals is provided to compare these algorithms with alternative speech enhancement algorithms, and to compare the performance of the iterative and sequential algorithms.

Active noise cancellation is an approach to noise reduction in which a secondary noise source that destructively interferes with the unwanted noise is introduced. In general, active noise cancellation systems rely on multiple sensors to measure the unwanted noise field and the effect of the cancellation. This paper develops an approach that utilizes a single sensor. The noise field is modeled as a stochastic process, and a time-adaptive algorithm is used to adaptively estimate the parameters of the process. Based on these parameter estimates, a canceling signal is generated. In general, the transfer function characteristics from the canceling source to the error sensor need to be accounted for. If these can be accurately measured in advance and are invertible except for the propagation delay between the source and sensor, then the essential problem becomes one of predicting future values of the noise field. The algorithm developed is evaluated with both artificially generated noise and with recordings of aircraft noise.>

We consider the problem of estimating the parameters of an unknown discrete linear system driven by a sequence of independent identically distributed (i.i.d.) random variables whose probability density function (PDF) may be non-Gaussian. We assume a general system structure that may contain causal and noncausal poles and zeros. The parameters characterizing the input PDF may also be unknown. We derive an asymptotic expression for the Cramer-Rao lower bound, and show that it is the highest (worst) in the Gaussian case, indicating that the estimation accuracy can only be improved when the input PDF is non-Gaussian. It is further shown that the asymptotic error variance in estimating the system parameters is unaffected by lack of knowledge of the PDF parameters, and vice verse. Computationally efficient gradient-based algorithms for finding the maximum likelihood estimate of the unknown system and PDF parameters, which incorporate backward filtering for the identification of non-causal parameters, are presented. The dual problem of blind deconvolution/equalization is considered, and asymptotically attainable lower bounds on the equalization performance are derived. These bounds imply that it is preferable to work with compact equalizer structures characterized by a small number of parameters as the attainable performance depend only on the total number of equalizer parameters.>

Identification of an unknown system and recovery of the input signals from observations of the outputs of an unknown multiple-input, multiple-output linear system are considered. Attention is focused on the two-channel case, in which the outputs of a 2*2 linear time invariant system are observed. The approach consists of reconstructing the input signals by assuming that they are statistically uncorrelated and imposing this constraint on the signal estimates. In order to restrict the set of solutions, additional information on the true signal generation and/or on the form of the coupling systems is incorporated. Specific algorithms are developed and tested. As a special case, these algorithms suggest a potentially interesting modification of Widrow's (1975) least-squares method for noise cancellation, where the reference signal contains a component of the desired signal.>

A class of iterative methods for solving the blind deconvolution problem, i.e. for recovering the input of an unknown possibly nonminimum-phase linear system by observation of its output, is presented. These methods are universal do not require prior knowledge of the input distribution, are computationally efficient and statistically stable, and converge to the desired solution regardless of initialization at a very fast rate. The effects of finite length of the data, finite length of the equalizer, and additive noise in the system on the attainable performance (intersymbol interference) are analyzed. It is shown that in many cases of practical interest the performance of the proposed methods is far superior to linear prediction methods even for minimum phase systems. Recursive and sequential algorithms are also developed, which allow real-time implementation and adaptive equalization of time-varying systems.>

The authors develop an approach to active noise cancellation using a single microphone. The noise field is modelled as a stochastic process, and a time-adaptive algorithm based on a modification of the block-estimate-maximize (EM) algorithm is used to adaptively estimate the parameters of this process. Based on these parameter estimates a canceling signal is generated. The algorithm developed is evaluated with recordings of aircraft noise, and has been implemented in real time with a single AT&T DSP32C chip.>

A necessary and sufficient condition for blind deconvolution (without observing the input) of nonminimum-phase linear time-invariant systems (channels) is derived. Based on this condition, several optimization criteria are proposed, and their solution is shown to correspond to the desired response. These criteria involve the computation only of second- and fourth-order moments, implying a simple tap update procedure. The proposed methods are universal in the sense that they do not impose any restrictions on the probability distribution of the (unobserved) input sequence. It is shown that in several important cases (e.g. when the additive noise is Gaussian), the proposed criteria are essentially unaffected.>

The authors propose a computationally efficient scheme for estimating the spatial (location) and spectral parameters of multiple-source signals using passive array data. The source signals are modeled as possibly correlated narrowband/wideband Gaussian random processes. The authors also allow sensor-to-sensor noise correlation. The proposed algorithm is optimal in the sense that it converges monotonically to the maximum-likelihood (ML) estimates (or, at least, to a stationary point of the likelihood function) of the unknown spatial and spectral parameters.>

The authors use methods of stochastic approximation to convert iterative algorithms for maximizing the Kullback-Liebler information measure (1959) into sequential algorithms. Special attention is given to the case of incomplete data, and a variety of algorithms are presented to deal with situations of that kind. The authors consider the application of these algorithms to the identification of finite-impulse-response (FIR) systems.>

A method is presented for evaluating the log-likelihood gradient (score), the Hessian, and the Fisher information matrix of the parameters of linear dynamic stochastic systems. The method incorporates the optimal Kalman smoothing equations and is therefore ideal for simultaneous state estimation and parameter identification. The result can be used for efficient implementation of gradient-based algorithms for maximum-likelihood identification of the unknown system parameters and for assessing the mean-square estimation accuracy.>

A class of sequential and adaptive algorithms for parameter estimation are presented that are based on the iterative estimate-maximize (EM) algorithm. In some cases sequential algorithms are derived that perform an exact EM step in each recursion; an example for these cases is given for the linear least-squares problem. In general, however, it is necessary to approximate the EM iteration in order to develop sequential algorithms. The application of this class of algorithms to the two-microphone noise cancellation problem is described.>

The estimate-maximize (EM) algorithm is an iterative method for finding maximum-likelihood parameter estimates from incomplete data. The authors develop an extension of the EM algorithm that may be useful in accelerating the algorithm and in simplifying the computations involved. The extension works with an intermediate complete data specification, and performs intermediate steps at some iterations. The authors consider the problem of parameter identification of a continuous-time linear dynamic system given discrete-time observations, and show that the proposed algorithm accelerates the convergence of the EM algorithm and simplifies the computations involved.>

A class of lower bounds on the mean-square error in parameter estimation is presented that are based on the Belini-Tartara lower bound (1974). The Wax-Ziv lower bound (1977) is shown to be a special case in the class. These bounds often are significantly tighter than the Chazan-Zakai-Ziv lower bound (1975) when the parameter to be estimated is subject to ambiguity and threshold effects.>

A general class of Bayesian lower bounds on moments of the error in parameter estimation is formulated, and it is shown that the Cramer-Rao, the Bhattacharyya, the Bobrovsky-Zakai, and the Weiss-Weinstein lower bounds are special cases in the class. The bounds can be applied to the estimation of vector parameters and any given function of the parameters. The extension of these bounds to multiple parameter is discussed.>

Single microphone speech enhancement systems have typically shown limited performance, while multiple microphone systems based on a least-squares error criterion have shown encouraging results in some contexts. In this paper we formulate a new approach to multiple microphone speech enhancement. Specifically, we formulate a maximum likelihood (ML) problem for estimating the parameters needed for canceling the noise in a two microphone speech enhancement system. This ML problem is solved via the iterative EM (Estimate-Maximize) technique. The resulting algorithm shows encouraging results when applied to the speech enhancement problem.

We present a computationally efficient scheme for parameter estimation of continuous dynamical linear systems given discrete-time noisy observations. The proposed scheme is optimal in the sense that it converges iteratively to the exact Maximum Likelihood estimate, where each iteration increases the likelihood.

A computationally efficient scheme for multi-path and multiple source location estimation, based on the EM algorithm is presented. The proposed scheme is optimal in the sense that it converges iteratively to the exact Maximum Likelihood of all source location parameters simultaneously.

We formulate a general class of lower bounds on the attainable mean square error (MSE) in parameter estimation, and show that the Cramer-Rao, the Bhattacharyya and the Bobrovsky-Zakai lower bounds are special cases in the class. We then propose a lower bound in the class that is often significantly tighter than the above mentioned bounds. The proposed bound is simple to analyze and compute; it is free from bias and regularity assumptions; it readily incorporates a priori information; and it readily generalizes to the estimation of vector parameters and any given function of the parameters.

We developed an algorithm for multiple source localization based on the Estimate-Maximize (EM) method. The EM method is an iterative algorithm that converges to the Maximum Likelihood (ML) estimate of the unknown parameters by exploiting the stochastic syctem under consideration. In our case the algorithm will converge to the exact ML estimates of the various sources location parameters, where each iteration increases the likelihood of those parameters.

A general class of lower bounds on the mean square error (mse) in random parameter estimation is formulated. These bounds are generated using functions of the parameter and the data that are orthogonal to the data. A particular choice in the class yields a new lower bound which is superior to both the Cramer-Rao and Bobrovsky-Zakai lower bounds.

A new lower bound on mean-square error in parameter estimation is presented. The bound is tighter than the Cramér-Rao and Bobrovsky-Zakai lower bounds. It requires no bias or regularity assumptions, it is computationally simple, and it can be applied to estimates of vector parameters or functions of the parameters.

The Ziv-Zakai Lower Bound (ZZLB) on the mean square error (m.s.e.) of time delay estimators is compared with theoretical and computer simulation results for time delay estimation via cross-correlation. Comparisons are made for both lowpass and narrowband signal spectra. For both signal spectra it is shown that for sufficiently large time-bandwidth product the correlator performance is very close to the ZZLB in both the small and large error regions. This establishes the cross-correlator as an optimal instrumentation (in the m.s.e. sense) while demonstrating that the ZZLB is an extremely tight lower bound. The ZZLB is further used to accurately predict the threshold signal-to-noise ratio (SNR) above which the cross-correlator performance is closely characterized by the Cramer-Rao Lower Bound (CRLB).

The location of a radiating source can be determined by measuring the relative time-delay of its signal wavefront to several spatially separated receivers. A new technique is presented to investigate the performance of time-delay measurement schemes based on a modified version of the Ziv-Zakai lower bound. This technique is shown to yield a tight bound on the attainable mean-square measurement error for any prespecified signal-to-noise ratio conditions.