The emergence of in-ear wearable sensors, so called Hearables, offers a convenient and cost-effective tool for long-term health monitoring. Traditionally, extracting frequency domain features from these signals relies on Power Spectral Density (PSD), which is highly susceptible to the effects of stochastic noise. This poses a formidable challenge for the processing of in-ear biosignals, which often have low signal-to-noise ratios (SNR). To address this problem, this work employs our recently proposed method for spectral analysis, termed "Panorama", defined as the Fourier transform of the autoconvolution. Experiments on in-ear EEG signals with extremely low SNR and in-ear sleep classification tasks show that, compared with the widely-used PSD, the Panorama spectrum offers better separability of features. We also show that a time series can be reconstructed from its autoconvolution, the inverse Fourier transform of Panorama, offering an additional analysis tool for time-domain applications.

Linear adaptive systems are a well-known staple in numerous signal processing applications. Recently, significant activity and performance gains have been achieved in multilayer neural networks for deep learning applied to practical data processing applications. In this paper, we describe the important relationships and significant differences between the procedures and methods used in linear adaptive systems and those used in multilayer neural networks for deep learning tasks. Input-output structures, cost functions and training criteria, adaptive algorithms, and data processing and optimization strategies are considered. It is the hope of the authors that this discussion will spur further crossover between the two fields, and in particular allow knowledge to be shared and further progress to be made.

A new quaternion-valued adaptive filtering algorithm based on extrapolated weight methods is proposed. The proposed algorithm belongs to the class of conjugate direction algorithms [1]. This class of extrapolation (momentum) based algorithms is preferred to RLS-based algorithms when the matrix inversion should be avoided, e.g. in the case of non-vector signals, sparse signals or non-stationary signals. This paper introduces Nesterov's optimal gradient methods in widely linear quaternion adaptive filtering. The resulting class of algorithm is shown to both have similar computational complexity and comparable performance to WLQRLS; however, the proposed method is more stable and outperforms WLQRLS in the non-stationary case.

This work sheds a new light on the spectral whitening effects of the sliding discrete Fourier transform (DFT) and uses it as a basis for a novel technique for circularity-preserving spectral estimation. This makes it possible to utilise full available spectral information, unlike the existing methods which ignore the phase spectrum. We then use the so introduced circularity-preserving DFT to show that the Wiener filter can be used to estimate the recently introduced second-order complementary spectral measure, termed the panorama, even in the critical cases of short data windows and incoherent sampling. Numerical examples demonstrate the ability of the proposed procedure to estimate the spectral circularity and the panorama, even in a streaming-data setting for which the current methods are inadequate.

Real world complex-valued signals typically exhibit rotation-dependent distributions (noncircularity), and significant performance gains in learning algorithms can be obtained by accounting for information beyond the standard second-order noncircularity (impropriety). To this end, we introduce a new closed form definition of complex correntropy which is general enough to cater for both circular and noncircular distributions in complex data, and serves as a basis for a novel cost function for widely linear adaptive filtering, termed the maximum improper complex corren-tropy criterion (MICCC). A stochastic gradient adaptive filtering algorithm is developed based on the MICCC, and its standard and complementary convergence and stability analyses are conducted with respect to both the circularity of the estimation error and the kernel size in the underlying Parzen estimator. Performance advantages over the strictly linear correntropy algorithm (MCCC) and the mean square error based complex least mean square (CLMS) and augmented CLMS (ACLMS) are demonstrated through analysis and simulations.

The least-mean-magnitude-phase (LMMP) algorithm is useful for complex-valued signal processing applications where precise control of magnitude and/or phase error information can provide improved estimation performance. Because it is a gradient procedure, however, the convergence speed of the algorithm can be limited for correlated input signals. In this paper, we derive affine-projection least-mean-magnitude-phase (AP-LMMP) algorithms based on an a posteriori update relation that have improved convergence performance over that of the LMMP algorithm without significant increases in complexity. We employ different nonlinear lookahead approaches depending on the projection order to compute the magnitudes of the a posteriori output signals and use these to implement the coefficient updates. Simulations indicate that AP-LMMP algorithms can outperform other algorithms in situations where their use is appropriate.

The Wiener filter is a well-known signal processing method for improving a noisy signal's quality. The Wiener filter requires either knowledge of or estimates of the power spectra of the signal-of-interest and of the undesired noise, leading to implementation challenges. In this paper, we show how a recently-developed second-order signal quantity termed the panorama can be employed to compute the Wiener filter for deterministic signals - containing nearly-constant frequency and phase components - in additive stochastic noise. We first show how the Wiener filter transfer function is related to the absolute value of the panorama and the power spectrum of the noisy measured signal. We then provide a practical procedure for estimating the absolute value of the panorama across frequency for single-channel sampled recordings. Numerical examples show the ability of the proposed procedure for estimating the panorama and for computing the Wiener filter for noisy deterministic signals.

The least-mean-magnitude-phase (LMMP) algorithm is useful for complex-valued signal processing applications where control of magnitude and/or phase error information is needed to achieve good overall performance. Due to the highly-nonlinear nature of the update terms in the LMMP and related methods, few convergence and stability results exist to guide step size choices for such algorithms. In this paper, we provide a rigorous stability and convergence analysis of the LMMP algorithm using robustness procedures and give sufficient stability conditions on the magnitude and phase step sizes to guarantee contraction-mapping behavior. We also provide an approximate relation for the steady-state MSE as a function of the step size values. Simulations verify the predictive powers of our analytical results and yield useful insights on step size choices in practice.

Widely linear processing has been shown to be superior to the traditional strictly linear processing in quaternion minimum mean square error (MMSE) estimation. However, a quantifiable performance difference between strictly and widely linear processing and the relationship between the performance and quaternion impropriety are still lacking. To this end, we present a proof for the performance advantage of widely linear estimation and relate the performance bounds to signal properties by exploiting the approximate joint diagonalisation of quaternion covariance matrices. In that sense, this work can be seen as a generalisation of complex-valued MMSE estimation, and can thus also be applied to the complex-valued case. Simulations on synthetic signals support the analysis.

In this paper, we develop a procedure for evaluating the performance of a single moving sensor system to estimate the range to a stationary emitter based on the discrete-time collection of bearing measurements along the trajectory travelled. We describe a numerical procedure to calculate the range p.d.f. from a chosen trajectory assuming a constant quality for the bearing measurements and use this procedure to evaluate the range root mean-square error as a function of the distance travelled. The logarithmic spiral family of trajectories is of particular interest, both from the standpoint of optimal control where such paths are derived, and from experimental biology, where large birds of prey are observed to travel in such paths in their search for food. Our performance analysis of these scenarios indicates why pitch angles less than 45° are to be preferred when a balanced range estimation performance throughout the trajectory is desired.

Current approaches to the mean-square analyses of the complex-least-mean-square (CLMS) and augmented CLMS (ACLMS) algorithms can be challenging due to the difficulty in diagonalising the augmented covariance matrix. By employing the recently introduced approximate uncorrelating transform (AUT), which diagonalizes the covariance and pseudocovariance matrices with a single singular value decomposition (SVD), we derive closed form expressions for both transient and steady-state mean square stability for the CLMS and ACLMS. Relationships between the degree of circularity of the input signal and the bound on the step-sizes of the CLMS and ACLMS are also established. We also show that for both CLMS and ACLMS, the steady-state misadjustment increases with the degree of non-circularity of the input signal. Simulations in the context of frequency estimation in power grid support the analyses.

The correlation preserving transform (CPT) is introduced to perform bivariate component analysis via decorrelating matrix decompositions, while at the same time preserving the integrity of original bivariate sources. Specifically, unlike existing bivariate uncorrelating matrix decomposition techniques, CPT is designed to preserve both the order of the data channels within every bivariate source and their mutual correlation properties. We introduce the notion of intraference to quantify the effects of interchannel mixing artifacts within recovered bivariate sources, and show that the integrity of separated sources is compromised when not accounting for the intrinsic correlations within bivariate sources, as is the case with current bivariate matrix decompositions. The CPT is based on augmented complex statistics and involves finding the correct conjugate eigenvectors associated with the pseudocovariance matrix, making it possible to maintain the physical meaning of the separated sources. The benefits of CPT are illustrated in the source separation and clustering scenarios, for both synthetic and real-world data.

In bearings-only geolocation, strategies that accurately combine both azimuth and elevation information for three-dimensional position estimation are required. This paper introduces a novel bearings-only geolocation algorithm, the Weighted Stansfield Algorithm in Three Dimensions (WS3D). This procedure uses a linearized approximation to the maximum likelihood cost for both the azimuth and elevation measurements, and uses both the bearing accuracies and estimates of the range to emitter in the zero-elevation geometric plane to minimize this cost in pair of least-squares procedures. Simulations of the new algorithm show that range and bearing accuracy weighting is important to obtain best performance. The algorithm also asymptotically outperforms a recently-derived instrumental variable (IV) algorithm for 3D geolocation designed to remove bias effects.

The autocorrelation does not differentiate between deterministic and stochastic signals, as phase information is not maintained. This paper introduces the autoconvolution for both deterministic and stochastic signals. The autoconvolution with the autocorrelation provides a second-order description that discriminates between deterministic and stochastic signals - even those with identical power spectra. We also introduce the panorama as the Fourier transform of the autoconvolution. The power spectrum and panorama admit a two-dimensional spectral representation that has unique and powerful properties, such as detecting deterministic sinusoidal components in correlated stochastic noise without knowledge of the sinusoidal frequencies or amplitudes. Additional extensions are indicated.

In multispectral and hyperspectral image analysis for remote sensing, variations in contrast due to cloud shadows and topography can cause problems in the demixing process, creating false endmembers and erroneous fractional abundance images. This paper introduces a novel hyperspectral mixing model in which pixel contrast is accounted for explicitly in the image formation. A method is described for estimating the per-pixel contrast for any chosen endmember-based demixing algorithm. Applications of the method to both synthetic and real-world satellite imagery illustrate its efficacy.

We introduce the concept of intra-ference in order to quantify the degree to which the integrity of bivariate (or complex) sources is preserved in applications based on matrix decompositions of bivariate data. This is achieved by examining the pseudocovariance matrix of noncircular complex sources, and by recognising that the pseudocovariance is intrinsically complex valued. We illuminate how the existing decompositions such as the strong uncorrelating transform (SUT) not only decorrelate the bivariate sources from one another, but also decorrelate and scatter the data channels within each bivariate source, thus violating source integrity. Examples showing that the intra-ference arises due to the phase ambiguity in the existing matrix decompositions support the approach.

This paper presents a family of least-squares algorithms for adaptive signal processing of complex-valued signals. The algorithms employ a composite cost function that allows magnitude and phase errors to be weighted differently in the parameter estimation depending on their importance, providing an opportunity for enhanced estimation performance over standard least-squares methods. We also describe a procedure for automatically adjusting this weighting based on the estimation errors themselves. Simulations show the excellent behavior of the algorithms in time-varying signal conditions.

This paper presents the least-mean-magnitude-phase (LMMP) algorithm for adaptive signal processing of complex-valued signals. The algorithm employs a decomposition of the mean squared error cost and allows different normalized step sizes to be selected for the magnitude and phase errors. Simulations show that the algorithm is useful when either the amplitude or the phase relationship between the input and desired response signals can be accurately estimated.

A new framework for the estimation of the instantaneous frequency in a three-phase power system is proposed. It is first illustrated that the complex-valued signal, obtained by the αβ transformation of three-phase power signals under unbalanced voltage sag conditions, is second order noncircular, for which standard complex adaptive estimators are suboptimal. To cater for second order noncircularity, an adaptive widely linear estimator based on the augmented complex least mean square (ACLMS) algorithm is proposed, and the analysis shows that this allows for optimal linear adaptive estimation for the generality of system conditions (both balanced and unbalanced). The enhanced robustness over the standard CLMS is illustrated by simulations on both synthetic and real-world voltage sags.

Howdy y'all! On behalf of the entire ICASSP 2010 Organizing Committee and the IEEE Signal Processing Society, it is my pleasure to welcome you to the 2010 IEEE International Conference on Acoustics, Speech, and Signal Processing in Dallas, Texas, U.S.A., held from March 14 to March 19, 2010. Our Conference is the 35th in a series of signal processing conferences dating back to 1976, when the first ICASSP was held in Philadelphia, PA. Our Conference is back in the United States after a highly-successful ICASSP 2009 in Taipei, Taiwan. Next year, ICASSP will travel to Central Europe, where it will be held in late May of 2011 in the beautiful jewel of a city that is Prague, Czech Republic. The tour that ICASSP takes from continent to continent underscores the international nature and global impact of the signal processing field. For those of you who are attending ICASSP from abroad, we especially welcome you after your long journey. No matter from where you have traveled, we hope that you find your stay in Dallas to be informative, invigorating, and most of all fun!

Recently, the augmented complex LMS (ACLMS) algorithm has been proposed for modeling complex-valued signal relationships in which a widely-linear model can be more appropriate. It is not clear, however, how the behavior of ACLMS differs from that of the conventional complex LMS (CCLMS) algorithm. In this paper, we leverage a recently-developed analysis for the complex LMS algorithm to illuminate the performance relationships between the ACLMS and CCLMS algorithms. Our analysis shows that the ACLMS algorithm can potentially achieve a lower steady-state mean-squared error as compared to that of CCLMS, but the convergence speed of ACLMS is slowed in the presence of highly non-circular complex-valued input signals. An adaptive beamforming example indicates the utility of the results.

Adaptive beamforming algorithms typically rely on a complex linear model between the sensor measurements and the desired signal output that does not enable the best performance from the data in some situations. In this paper, we present an extension of the well-known recursive least-squares algorithm for adaptive filters to widely-linear complex-valued signal and system modeling. The widely-linear RLS algorithm exploits a structured covariance matrix update that maintains information about the non-circularity of the input data to solve the widely-linear least-squares task at each snapshot. In addition, the WL-RLS algorithm can easily be switched between conventional and widely-linear complex modeling as needed. Application of the method to adaptive beamforming of mixed BPSK and QPSK signal transmissions shows that the system can extract all of the transmitted signal outputs in certain overloaded scenarios, and it performs up to 3 dB better than the conventional RLS beamformer when the array is not overloaded.

We address the duality between adaptive filtering in C and R2and provide a comparison between the well understood dual channel real valued least mean square (DCRLMS) algorithm in R2and the corresponding algorithms in C. These include the complex LMS (CLMS) and the recently introduced augmented CLMS (ACLMS), a widely linear algorithm designed for the processing of noncircular complex valued signals. The analysis shows that the standard CLMS and DCRLMS in general provide different adaptive filtering solutions, whereas the ACLMS and DCRLMS are isomoprhic and can be made equivalent. The analysis is supported by simulations on noncircular real world signals.

We present a new spatio-temporal algorithm for speech enhancement using microphone arrays. Our technique uses an iterative method for computing the generalized eigenvectors of the multichannel data as measured from the microphone array. Coefficient adaptation is performed using the spatio-temporal correlation coefficient sequences of the observed data. The technique avoids large matrix-vector multiplications and has lower computational resource requirements as compared to competing methods. The technique also does not require a calibrated microphone array and is applicable to a wide variety of noise types, including stationary correlated noise and nonstationary speech-like (e.g., babble) background noise. Application of the method to microphone array data in various environmental settings indicate that the procedure can achieve significant gains in signal-to-interference ratios (SIRs) even in low SIR environments, without introducing musical tone artifacts in the enhanced speech.

In both the natural gradient algorithm and the FastICA algorithm for blind source separation (BSS), output nonlinearities for each extracted source must be selected, and the performance of each approach can be sensitive to the chosen output nonlinearity. In this paper, we propose to use simple piecewise-linear output nonlinearities for these algorithms and obtain a number of useful properties with such a choice. For the natural gradient BSS algorithm, nonlinearity- switching is easily achieved through a common stability criterion that guarantees local stability for all source distributions. For the FastICA algorithm, the chosen nonlinearities can be very close to linear, suggesting that simple (e.g. mu-law) output companding is sufficiently nonlinear to allow separation when used with this algorithm. Simulations are provided to verify the theoretical results.

We present anew spatio-temporal algorithm for speech enhancement using microphone arrays. Our technique involves the development of an iterative method for computing the generalized eigenvectors of the multichannel data as measured from the microphone array. Coefficient adaptation is performed using the spatio-temporal correlation coefficient sequences of the observed data. The technique avoids large matrix-vector multiplications; hence, the computation time and computational resource requirements are significantly smaller than those of competing methods. The technique is applicable to a wide variety of noise types, including stationary correlated noise and non- stationary speech-like (babble) background noise, without requiring any noise-dependent parameter settings.

Using intelligent resource allocation based on the information content in the imaging system's field-of-view for the successful design of a flat-profile multiplexed optical imaging system requires the use of adaptive techniques. This paper describes a model-based technique for determining regions of interest in aerial images using the 2D normalized power spectral density within Gilles' saliency map estimator. The proposed technique exploits the 1/falphaspatial spectral shape of such natural imagery in a computationally-simple approach that is robust to additive noise. Application of the method to candidate aerial images shows its ability to identify consistent regions of interest for such data.

The importance of mathematics and science education in today's modern, technology-driven society cannot be understated. This paper describes the history of and ongoing efforts in curricular and educational technology development for the infinity project, a joint effort between educators, administrators, and industry leaders to establish an engineering curriculum at the high school level. Several issues are considered, including the choice and design of the technology platform used in the curriculum, the curricular and technology development timeline, and the design examples chosen to illuminate important engineering concepts within the curriculum. We also provide a list of key best practices and challenges in developing novel pre-college engineering curricula for future engineers.

Algorithms for adapting orthogonal matrices in optimization and signal processing typically employ the geometry of either the Grassmann manifold or the Stiefel manifold depending on the chosen cost function. In this paper, we develop gradient adaptive algorithms that use the geometry of both manifolds in their operation. Such algorithms offer a straightforward way to mitigate numerical error accumulation due to discretization of the coefficient updates. Examples drawn from subspace tracking and eigenvector analysis illustrate the usefulness of the design methods.

This paper describes a novel modification to the well-known natural gradient or INFOMAX algorithm for blind source separation that largely mitigates its divergence problems. The modified algorithm imposes an a posteriori scalar gradient constraint that adds little computational complexity to the algorithm and exhibits fast convergence and excellent performance for fixed step size values that are largely independent of input signal magnitudes and initial separation matrix estimates. Evaluation of the approach for the separation of instantaneous and convolutive source mixtures using both time- and frequency-domain implementations shows its excellent separation behavior.

Successful design of a flat-profile multiplexed optical imaging system requires the use of adaptive techniques to make intelligent resource allocation based on the information content in the imaging system's field-of-view. This paper explores techniques for finding regions of interest in aerial images using local entropy as a descriptor. A novel method for identifying regions-of-interest in images is developed using the 2D normalized power spectral density within Gilles' saliency map estimator. Application of the method to candidate aerial images shows its ability to identify consistent regions of interest for such data for varying block sizes and under additive noise.

The performance of any signal enhancement method depends on the relative amplitudes of the signals and interferences present in the original measurements. Previous evaluations of convolutive blind source separation methods for speech enhancement have considered situations where the signal-to-interference ratios (SIRs) of all the talkers' speech signals are nearly equal in the recorded signal mixtures. This paper presents real-world separation experiments of two- and three-talker signal mixtures in which the level of one talker's speech is lower than those of the others. Methods based on decorrelation, frequency-domain information maximization, and time-domain contrast optimization are studied. Experimental evaluation shows that the weaker talker's speech receives the most enhancement in the separated system's outputs, whereas the stronger talker's speech signals receive moderate to little enhancement beyond a limiting value.

This paper derives two spatio-temporal extensions of the well-known FastICA algorithm of Hyvarinen and Oja that are applicable to the convolutive blind source separation task. Our time-domain algorithms combine multichannel spatio-temporal prewhitening via multistage least-squares linear prediction with novel adaptive procedures that impose paraunitary constraints on the multichannel separation filter. The techniques converge quickly to a separation solution without any step size selection or divergence difficulties, and unlike other methods, ours do not require special coefficient initialization procedures to obtain good separation performance. They also allow for the efficient reconstruction of individual signals as observed in the sensor measurements directly from the system parameters for single-input multiple-output blind source separation tasks. An analysis of one of the adaptive constraint procedures shows its fast convergence to a paraunitary filter bank solution. Numerical evaluations of the proposed algorithms and comparisons with several existing convolutive blind source separation techniques indicate the excellent relative performance of the proposed methods.

In blind source separation and independent component analysis, it is desirable to select a separation criterion that results in a simple algorithm and achieves accurate and robust source estimates. In this paper, we propose to use the Huber M-estimator cost function as the contrast function within the FastICA algorithm of Hyvärinen and Oja. The algorithm obtained from this cost is particularly simple to implement. We establish key properties regarding the local stability of the algorithm for general non-Gaussian source distributions, and its separating capabilities are shown through analysis to be largely insensitive to the cost function’s threshold parameter. Simulations comparing the performance of this algorithm to standard FastICA implementations are given.

In some signal processing tasks involving complex-valued multichannel measurements, classical whitening approaches do not completely remove the second-order statistical dependencies of the data. This paper describes adaptive procedures for estimating the strong uncorrelating transform for jointly diagonalizing the covariance and pseudo-covariance matrices of multidimensional signals. Novel algorithms are derived that extend and combine the power method and orthogonal iterations with ordinary fixed and iterative whitening procedures. Finally, we show how to combine our procedures with orthogonal PAST algorithms to perform subspace tracking and source signal clustering based on non-circularity

The paper presents a spatio-temporal extension of the well-known fastICA algorithm of Hyva/spl uml/rinen and Oja that is applicable to both convolutive blind source separation and multichannel blind deconvolution tasks. Our time-domain algorithm combines multichannel spatio-temporal prewhitening via multi-stage least-squares linear prediction with a fixed-point iteration involving a new adaptive technique for imposing paraunitary constraints on the multichannel separation filter. Our technique also allows for efficient reconstruction of individual signals as observed in the sensor measurements for single-input, multiple-output (SIMO) BSS tasks. Analysis and simulations verify the utility of the proposed methods.

We propose a novel method for the optimal co-design of the optical and reconstruction filters in a computational imaging system. Closed form solutions are presented for the design of an optimal observation matrix for a fixed reconstruction matrix. An iterative method for computing global optimal filters is then proposed based on the derived analytical solution and the well known Wiener filter. The performance of the proposed optimal filters represents a universal bound on the performance of any physically realizable computational imaging system.

Natural gradient adaptation is an especially convenient method for adapting the coefficients of a linear system in inverse filtering tasks such as convolutive blind source separation and multichannel blind deconvolution. When developing practical implementations of such methods, however, it is not clear how best to window the signals and truncate the filter impulse responses within the filtered gradient updates. We show how inadequate use of truncation of the filter impulse responses and signal windowing within a well-known natural gradient algorithm for multichannel blind deconvolution and source separation can introduce a bias into its steady-state solution. We then provide modifications of this algorithm that effectively mitigate these effects for estimating causal FIR solutions to single- and multichannel equalization and source separation tasks. The new multichannel blind deconvolution algorithm requires approximately 6.5 multiply/adds per adaptive filter coefficient, making its computational complexity about 63% greater than the originally-proposed version. Numerical experiments verify the robust convergence performance of the new method both in multichannel blind deconvolution tasks for i.i.d. sources and in convolutive BSS tasks for real-world acoustic sources, even for extremely-short separation filters.

In wireless communications, it is often desirable to merge bit decisions from multiple receivers to improve overall link performance. It is well known that in order to fuse bit decisions optimally from a network of receivers, precise knowledge of receiver bit error rates (BERs) is needed. This information, however, is rarely available in practice. We present an iterative procedure for blindly estimating receiver BERs to enable near optimal blind fusion of bit decisions in a multi-receiver network. We show that the solution of the estimation problem is a structured eigenvalue task and propose a modified power method procedure to perform it. We prove that the desired solution is a stable point of the algorithm and the algorithm is locally stable. Furthermore, we show, via simulations, that the technique results in excellent performance in nearly all practical operating scenarios.

Practical gradient-based adaptive algorithms for multichannel blind deconvolution and convolutive blind source separation typically employ FIR filters for the separation system. Inadequate use of signal truncation within these algorithms can introduce steady-state biases into their converged solutions that lead to degraded separation and deconvolution performances. We derive a natural gradient multichannel blind deconvolution and source separation algorithm that mitigates these effects for estimating causal FIR solutions to these tasks. Numerical experiments verify the robust convergence performance of the new method both in multichannel blind deconvolution tasks for i.i.d. sources and in convolutive BSS tasks for acoustic sources, even for extremely-short separation filters.

We present an adaptive algorithm that blindly separates mixtures of finite-alphabet sources given knowledge of the source alphabet and distribution. The algorithm is a nonlinear recursive least-squares procedure that employs a simple and numerically-robust square root Householder update. Simulations verify that the algorithm can separate large-scale noisy mixtures of finite-alphabet sources without any knowledge of the number of sources in the mixture.

In this paper, we describe a simple gradient algorithm for adapting the phase response of an FIR filter whose magnitude response is already specified. The algorithm is an extension of a previously-developed gradient adaptive algorithm for allpass filtering. The algorithm directly adjusts the impulse response of the FIR filter using input and desired response signals in a simple manner without the need for frequency-domain processing or coefficient monitoring. A stationary point analysis of the algorithm verifies its desirable estimation capabilities. Simulations confirm the capability of the algorithm in a filter modeling task.

Many blind source separation (BSS) methods are based on minimization of entropy-based criteria. Gradient-based methods have been observed to converge slowly when applied to such criteria. In this paper, we argue that the non-convex “bird's beak” shape of such criteria is the reason why gradient BSS methods perform poorly. To overcome these limitations, we propose a novel local parameter optimization method based upon curve fitting to a density-based entropy measure. Simulations show that the novel method can be used as an effective refinement procedure for an existing BSS method.

The Infinity Project is a joint effort between university educators, high school teachers, administrators, and industrial leaders to establish an engineering curriculum that is taught within the regular high school day. The curriculum teaches students about the design of technology-driven systems and motivates them to learn fundamental concepts of mathematics, science, and engineering. The curriculum currently consists of (1) a course text [1], (2) integrated laboratory exercises with real-time signal processing hardware, (3) summer teacher training institutes, and (4) a web community portal for information sharing (www.infinity-project.org). In this paper, we briefly describe three real-time signal processing laboratory exercises—SketchWave, Coin Counter, and Video Steganography—that are used within the course to both teach and excite students about engineering design, fundamental scientific and mathematical concepts, and the power of problem solving. These representative exercises are drawn from over 200 laboratory worksheets developed for the course and available online for participating schools.

This paper introduces a class of normalized natural gradient algorithms (NNGs) for adaptive filtering tasks. Natural gradient techniques are useful for generating relatively simple adaptive filtering algorithms where the space of the adaptive coefficients is curved or warped with respect to Euclidean space. The advantage of normalizing gradient adaptive filters is that constant rates of convergence for signals with wide dynamic ranges may be achieved. We show that the so-called proportionate normalized least mean squares (PNLMS) algorithm, an adaptive filter that converges quickly for sparse solutions, is in fact an NNG on a certain parameter space warping. We also show that by choosing a warping that favors diverse or dense impulse responses, we may obtain a new adaptive algorithm, the inverse proportionate NLMS (INLMS) algorithm. This procedure converges quickly to and accurately tracks non-sparse impulse responses.

Many blind deconvolution algorithms have been designed to extract digital communications signals corrupted by inter-symbol interference. Such algorithms generally fail when applied to signals with impulsive characteristics, such as acoustic signals. In this paper, we provide a theoretical analysis and explanation as to why Bussgang-type algorithms are generally unsuitable for deconvolving impulsive signals. We then propose a novel modification of one such algorithm, the Sato algorithm, to enable it to deconvolve such signals. Sufficient conditions on the source signal to guarantee local stability of the modified Sato algorithm about a deconvolving solution are derived. Computer simulations show the efficiency of the proposed approach as compared to the Shalvi-Weinstein algorithm for deconvolving impulsive signals.

Time delay estimation (TDE) is an important procedure for many communications and signal processing tasks. In this paper, a novel TDE algorithm is proposed. The block-based method estimates the phase response of a sine-interpolation filter using Newton's method on the space of allpass filters, from which the delay estimate is calculated using a simple coefficient-based formula. The algorithm is amenable to fast convolution procedures and needs only a few iterations to converge. Simulations comparing the proposed method with the well-known linear phase and time-domain cross-correlation methods show that our technique achieves a lower mean-squared error than these other methods in low signal-to-noise ratio environments.

The search for universally applicable nonlinearities in blind signal separation has produced nonlinearities that are optimal for a given distribution, as well as nonlinearities that are most robust against model mismatch. This paper shows yet another justification for the score function, which is in some sense a very robust nonlinearity. It also shows that among the class of parameterizable nonlinearities, the threshold nonlinearity with the threshold as a parameter is able to separate any non-Gaussian distribution, a fact that is also proven in this paper.

This paper outlines the goals, structure, and technology elements of the INFINITY Project, a joint effort between university educators, industrial partners, and civic leaders to introduce a signal-processing-based engineering curriculum at the high school educational level. Implementation issues of the program are addressed, including class textbook and laboratory creation, teacher training, and online classroom support. Initial responses to the effort from teachers and students at fourteen different high schools are highly positive, and plans for further expansion of the program are given.

We present novel algorithms for multichannel blind deconvolution under output whitening constraints. The algorithms are inspired by recently-developed procedures for gradient adaptive paraunitary filter banks. Several algorithms are developed, including one algorithm that successfully deconvolves mixtures of arbitrary non-zero kurtosis source signals. We provide detailed local stability analyses of the proposed methods to verify their capabilities. Simulations show that the methods successfully deconvolve spatio-temporal mixtures of statistically independent source signals.

We derive two new O(N/sup 2/) algorithms for arbitrary recursive least-squares (RLS) estimation tasks. The algorithms employ a novel update for an inverse square-root factor of the exponentially-windowed input signal autocorrelation matrix that is the least-squares equivalent of a natural gradient prewhitening algorithm. Both of the new RLS algorithms require 4N/sup 2/+O(N) multiply/adds, two divides, and one square root per iteration to implement. We can prove that our new algorithms are numerically-robust, and simulations are used to indicate this fact in fixed-point arithmetic. An algorithm that computes the square-root factorization of the input signal autocorrelation matrix is also described.

We propose a blind signal separation algorithm (CLUE) that uses the sum of the individual code lengths of the extracted signals as a measure of the separation performance. The new technique combines a widely-available universal data compression routine with any single-parameter search procedure. Unlike previous approaches, the proposed method is model-free and does not rely on the moment values of the signals for its separation performance. An example shows the algorithm's efficiency in separating mixtures of image, audio, and text data.

In this paper, we present a simple adaptive filtering solution to general phase equalization and phase modeling tasks. Our proposed algorithm maintains approximate allpass constraints on the coefficients of an adaptive FIR filter in a self-stabilized manner. An analysis of an idealized IIR version of the technique shows that it converges to a unique stationary point corresponding to the optimal phase relationship between the processed signals. The relationship between the mean-squared error of the estimated phase response and the mean-squared error of the filter coefficients is also derived. Simulations verify that the proposed algorithm performs accurate phase equalization and approximation of unknown systems even in low signal-to-noise ratio conditions.

Recently, developments in self-stabilized algorithms for gradient adaptation of orthonormal matrices have resulted in simple but powerful principal and minor subspace analysis methods. In this paper, we extend these ideas to develop algorithms for instantaneous prewhitened blind separation of homogeneous signal mixtures. Our algorithms are proven to be self-stabilizing to the Stiefel manifold of orthonormal matrices, such that the rows of the adaptive demixing matrix do not need to be periodically reorthonormalized. Several algorithm forms are developed, including those that are equivariant with respect to the prewhitened mixing matrix. Simulations verify the excellent numerical properties of the proposed methods for the blind source separation task.

Paraunitary filter banks are important for several signal processing tasks. We consider the task of adapting the coefficients of a multichannel FIR paraunitary filter bank via gradient ascent or descent on a chosen cost function. The proposed generalized algorithms inherently adapt the system's parameters in the space of paraunitary filters. Modifications and simplifications of the techniques for spatio-temporal principal and minor subspace analysis are described. Simulations verify one algorithm's useful behavior in this task.

In some situations where active noise control could be used, the well-known multichannel version of the filtered-X least mean square (LMS) adaptive filter is too computationally complex to implement. We develop a fast, exact implementation of this adaptive filter for which the system's complexity scales according to the number of filter coefficients within the system. In addition, we extend computationally efficient methods for effectively removing the delays of the secondary paths within the coefficient updates to the multichannel case, thus yielding fast implementations of the LMS adaptive algorithm for multichannel active noise control. Examples illustrate both the equivalence of the algorithms to their original counterparts and the computational gains provided by the new algorithms.

Gradient adaptation is a useful technique for adjusting a set of parameters to minimize a cost function. While often easy to implement, the convergence speed of gradient adaptation can be slow when the slope of the cost function varies widely for small changes in the parameters. In this paper, we outline an alternative technique, termed natural gradient adaptation, that overcomes the poor convergence properties of gradient adaptation in many cases. The natural gradient is based on differential geometry and employs knowledge of the Riemannian structure of the parameter space to adjust the gradient search direction. Unlike Newton's method, natural gradient adaptation does not assume a locally-quadratic cost function. Moreover, for maximum likelihood estimation tasks, natural gradient adaptation is asymptotically Fisher-efficient. A simple example illustrates the desirable properties of natural gradient adaptation.

We consider the problem of selective transmission-the dual of the blind source separation task-in which a set of independent source signals are adaptively premixed prior to a non-dispersive physical mixing process so that each source can be independently monitored in the far field. We derive a stochastic gradient algorithm for iteratively-estimating the premixing matrix in the selective transmission problem, and through a simple modification, we obtain a second algorithm whose performance is equivariant with respect to the channel's mixing characteristics. We also describe an approximate version of the equivariant algorithm and other implementation issues. Simulations indicate the useful behavior of the premixing algorithms for selective transmission.

In this letter, we present a minor subspace rule that extracts the subspace that spans the m minor components of a n-dimensional vector stationary random process, m

In some situations where active noise control could be used, the well-known multichannel version of the filtered-X LMS adaptive filter is too computationally-complex to implement. In this paper, we develop a fast, exact implementation of this multichannel system whose complexity is approximately O(2L) per filter channel, where L is the FIR filter length. In addition, we provide a computationally-efficient method for effectively removing the delays of the secondary paths within the coefficient updates, thus yielding a fast implementation of the LMS adaptive algorithm for multichannel active noise control. Examples illustrate both the equivalence of the algorithms to their original counterparts and the computational gains provided by the new algorithms.

Recently, two simple gradient-based algorithms for unbiased IIR system identification in the presence of zero-mean correlated output noise were derived and shown to perform well in simulation. In this paper, we study the stability and robustness of these two adaptive filters, deriving strictly positive real (SPR) conditions on the overall unknown-plus-adaptive systems to guarantee convergence of the coefficients to their optimum values. Unlike other algorithms for unbiased IIR adaptive filtering, the stability of each of these algorithms depends on the initial values of the filter coefficients. However, near the optimum coefficient solutions, both algorithms are locally-stable, irrespective of the unknown system. Simulations verify the results of our analyses.

Past methods for mapping the least-mean-square (LMS) adaptive finite-impulse-response (FIR) filter onto parallel and pipelined architectures either introduce delays in the coefficient updates or have excessive hardware requirements. In this paper, we describe a pipelined architecture for the LMS adaptive FIR filter that produces the same output and error signals as would be produced by the standard LMS adaptive filter architecture without adaptation delays. Unlike existing architectures for delayless LMS adaptation, the new architecture's throughput and hardware complexity are independent of and linear with the filter length, respectively.

The filtered-X LMS algorithm used in active noise control applications suffers from slow convergence due in part to the output delay caused by the secondary path of the system. In this letter, we provide a computationally efficient method for removing this output delay within the error signal used to update the controller coefficients, such that the standard LMS algorithm is effectively employed.

The recently-introduced fast affine projection (FAP) adaptive filter has performance rivalling that of the RLS adaptive filter in many situations. However, the sliding-window RLS algorithm embedded within the FAP adaptive filter can be too complex to implement in some cases. We describe an alternate implementation of the FAP adaptive filter that uses orthogonal transforms to approximately calculate the affine projection. Simulations show that the performance of this simplified algorithm is nearly as good as the original algorithm when the transform is chosen properly. In addition, we provide a gradient step size (GSS) method for enhancing the performance of the FAP adaptive filter in nonstationary signal environments.

We derive new adaptive step size algorithms for two general classes of modified stochastic gradient adaptive filters that include the sign-error, sign-data, sign-sign, and normalized gradient adaptive filters as specific cases. These computationally-simple parameter adjustment algorithms are based on stochastic gradient approximations of steepest descent procedures for the unknown parameters. Analyses of the algorithms show that the stationary points of the steepest descent procedures yield the optimum step size values at each time instant as obtained from statistical analyses of the adaptive filter updates. Simulations verify the theoretical results and indicate that near-optimal tracking performance can be obtained from each of the adaptive step size algorithms without any knowledge of the rate of change of the unknown system.

Stochastic-gradient adaptive algorithms for block-based or frequency-domain single channel filters and for true multichannel filters share a common multiple-error LMS algorithm update, given by W/sub k+1/=W/sub k/+/spl mu/X/sub k/(D/sub k/-X/sub k//sup T/W/sub k/). We examine the mean-square performance of the multiple-error LMS adaptive algorithm for correlated Gaussian input data channels and arbitrary i.i.d. input data channels. Using our analysis, we show that the multiple-error LMS algorithm performs uniformly worse than the single-channel LMS algorithm for a given amount of data consumed. We also derive simple step size bounds to guarantee mean-square convergence of the multiple-error and block LMS adaptive algorithms. Simulations of both the block LMS adaptive algorithm and the multichannel filtered-X LMS adaptive algorithm corroborate our theoretical results.>

A derivation of the normalized LMS algorithm is generalized, resulting in a family of projection-like algorithms based on an L/sub p/-minimized filter coefficient change. The resulting algorithms include the simplified NLMS algorithm of Nagumo and Noda (1967) and an even simpler single-coefficient update algorithm based on the maximum absolute value datum of the input data vector. A complete derivation of the algorithm family is given, and simulations are performed to show the convergence behaviors of the algorithms.>

In almost all analyses of the LMS adaptive filter, it is assumed that the filter coefficients are statistically independent of the input data currently in filter memory, an assumption which is incorrect for shift input data. A method of generating an exact statistical description of the mean and mean-square convergence of the LMS algorithm with shift input data without using this assumption is described. Given input data that are independent from sample to sample, iterations are developed for expectations of products of statistically dependent filter coefficients and data samples, from which important quantities such as the excess mean-square error can be found. Monte Carlo simulations show that the exact analysis produced by this method predicts the mean-square convergence much more accurately than the analysis with the independence assumption for large step sizes, and this accuracy is maintained throughout the useful step size range of the algorithm. With this analysis, phenomena due to the coupling of filter coefficients and past data are discovered and verified.>

The authors prove that for zero-mean white data input, the optimum algorithm which modifies the data vector in the LMS (least mean square) gradient estimate to achieve the lowest excess mean-square error for a given convergence rate is the normalized LMS (NLMS) algorithm. It is shown that this adaptive filtering algorithm is equivalent to recursive least-squares adaptation with a known diagonal data covariance matrix. Moreover, the algorithm can be interpreted as a modified LMS algorithm in which the iterated weight vector is used to form the error estimate. Both theoretical calculations and simulations for white Gaussian data show that this NLMS algorithm performs as much as 3.6 dB better than standard LMS for the input.>

An examination is made of the effect of a memoryless nonlinearity acting upon the error in LMS (least-mean-square) adaptation. Results for E(e/sup 2K/) minimization are extended to general nonlinear error adaptation, and equations expressing bounds on step size, time constants, and misadjustment are derived. A general performance factor expressing the improvement in misadjustment over standard LMS adaptation for a given convergence rate is presented. Using the calculus of variations, it is shown that the optimum nonlinearity to minimize misadjustment is -p'(x)/p(x), where p(x) is the probability density function of the uncorrelated plant noise. Comparisons of this result with the Cramer-Rao bound indicate that choice of this nonlinearity yields an asymptotically optimal stochastic gradient algorithm. Simulations verify the result that the optimum nonlinearity for minimizing misadjustment for Laplacian plant noise is sgn(x), and a 3-dB improvement is obtained under these conditions.>