Most studies of adaptive algorithm behavior consider performance measures based on mean values such as the mean value of the squared error. Behavior models based on average measures models are useful for understanding the algorithm behavior under different environments and can be used for design. Nevertheless, from a practical point of view, the adaptive filter user has only one realization of the algorithm to obtain the desired result. This article derives a model for the variance of the squared-error sample curve of the least-mean-square (LMS) adaptive algorithm, so that the achievable cancellation level can be predicted based on the properties of the steady-state squared error. The derived results provide the user with useful design guidelines.

In hyperspectral sparse unmixing, a successful approach employs spectral bundles to address the variability of the endmembers in the spatial domain. However, the regularization penalties usually employed aggregate substantial computational complexity, and the solutions are very noise-sensitive. We generalize a multiscale spatial regularization approach to solve the unmixing problem by incorporating group sparsity-inducing mixed norms. Then, we propose a noise-robust method that can take advantage of the bundle structure to deal with endmember variability while ensuring inter- and intra-class sparsity in abundance estimation with reasonable computational cost. We also present a general heuristic to select themost representativeabundance estimation over multiple runs of the unmixing process, yielding a solution that is robust and highly reproducible. Experiments illustrate the robustness and consistency of the results when compared to related methods.

The recent evolution of hyperspectral imaging technology and the proliferation of new emerging applications press for the processing of multiple temporal hyperspectral images. In this work, we propose a novel spectral unmixing (SU) strategy using physically motivated parametric endmember (EME) representations to account for temporal spectral variability. By representing the multitemporal mixing process using a state-space formulation, we are able to exploit the Bayesian filtering machinery to estimate the EME variability coefficients. Moreover, by assuming that the temporal variability of the abundances is small over short intervals, an efficient implementation of the expectation–maximization (EM) algorithm is employed to estimate the abundances and the other model parameters. Simulation results indicate that the proposed strategy outperforms state-of-the-art multi-temporal SU (MTSU) algorithms.

Autoencoder (AEC) networks have recently emerged as a promising approach to perform unsupervised hyperspectral unmixing (HU) by associating the latent representations with the abundances, the decoder with the mixing model, and the encoder with its inverse. AECs are especially appealing for nonlinear HU since they lead to unsupervised and model-free algorithms. However, existing approaches fail to explore the fact that the encoder should invert the mixing process, which might reduce their robustness. In this letter, we propose a model-based AEC for nonlinear HU by considering the mixing model a nonlinear fluctuation over a linear mixture. Different from previous works, we show that this restriction naturally imposes a particular structure to both the encoder and decoder networks. This introduces prior information in the AEC without reducing the flexibility of the mixing model. Simulations with synthetic and real data indicate that the proposed strategy improves nonlinear HU.

In this paper, we propose to jointly solve the hyperspectral super-resolution problem and the unmixing problem of the underlying super-resolution image using a coupled LL1 block-tensor decomposition. We consider a spectral variability phenomenon occurring between the observed low-resolution images. Exact recovery conditions for the image and mixing factors are provided. We propose two algorithms, an unconstrained one and another one subject to nonnegativity constraints, to solve the problems at hand. We showcase performance of the proposed approach on synthetic and real images.

Several approaches have been proposed to solve the spectral unmixing problem in hyperspectral image analysis. Among them the use of sparse regression techniques aims to characterize the abundances in pixels based on a large library of spectral signatures known a priori. Recently, the integration of image spatial-contextual information significantly enhanced the performance of sparse unmixing. In this work, we propose a computationally efficient multiscale representation method for hyperspectral data adapted to the unmixing problem. The proposed method is based on a hierarchical extension of the SLIC oversegmentation algorithm constructed using a robust homogeneity testing. The image is subdivided into a set of spectrally homogeneous regions formed by pixels with similar characteristics (superpixels). This representation is then used to provide prior spatial regularity information for the abundances of materials present in the scene, improving the conditioning of the unmixing problem. Simulation results illustrate that the method is capable of estimating abundances with high quality and low computational cost, especially in noisy scenarios.

Multiple endmember spectral mixture analysis (MESMA) is one of the leading approaches to perform spectral unmixing (SU) considering the variability of the endmembers (EMs). It represents each EM in the image using libraries of spectral signatures acquireda priori. However, existing spectral libraries are often small and unable to properly capture the variability of each EM in practical scenes, which compromises the performance of MESMA. In this letter, we propose a library augmentation strategy to increase the diversity of existing spectral libraries, thus improving their ability to represent the materials in real images. First, we leverage the power of deep generative models to learn the statistical distribution of the EMs based on the spectral signatures available in the existing libraries. Afterward, new samples can be drawn from the learned EM distributions and used to augment the spectral libraries, improving the overall quality of the SU process. Experimental results using synthetic and real data attest to the superior performance of the proposed method even under library mismatch conditions.

Traditional hyperspectral unmixing methods neglect the underlying variability of spectral signatures often observed in typical hyperspectral images (HI), propagating these mismodeling errors throughout the whole unmixing process. Attempts to model material spectra as members of sets or as random variables tend to lead to severely ill-posed unmixing problems. Although parametric models have been proposed to overcome this drawback by handling endmember (EM) variability through generalizations of the mixing model, the success of these techniques depends on employing appropriate regularization strategies. Moreover, the existing approaches fail to adequately explore the natural multidimensinal representation of HIs. Recently, tensor-based strategies considered low-rank decompositions of HIs as an alternative to impose low-dimensional structures on the solutions of standard and multitemporal unmixing problems. These strategies, however, present two main drawbacks: 1) they confine the solutions to low-rank tensors, which often cannot represent the complexity of real-world scenarios and 2) they lack guarantees that EMs and abundances will be correctly factorized in their respective tensors. In this article, we propose a more flexible approach, called unmixing with low-rank tensor regularization algorithm accounting for EM variability (ULTRA-V), that imposes low-rank structures through regularizations whose strictness is controlled by scalar parameters. Simulations attest the superior accuracy of the method when compared with state-of-the-art unmixing algorithms that account for spectral variability.

Image fusion combines data from different heterogeneous sources to obtain more precise information about an underlying scene. Hyperspectral-multispectral (HS-MS) image fusion is currently attracting great interest in remote sensing since it allows the generation of high spatial resolution HS images and circumventing the main limitation of this imaging modality. Existing HS-MS fusion algorithms, however, neglect the spectral variability often existing between images acquired at different time instants. This time difference causes variations in spectral signatures of the underlying constituent materials due to the different acquisition and seasonal conditions. This paper introduces a novel HS-MS image fusion strategy that combines an unmixing-based formulation with an explicit parametric model for typical spectral variability between the two images. Simulations with synthetic and real data show that the proposed strategy leads to a significant performance improvement under spectral variability and state-of-the-art performance otherwise.

Spectral variability in hyperspectral images can result from factors including environmental, illumination, atmospheric and temporal changes. Its occurrence may lead to the propagation of significant estimation errors in the unmixing process. To address this issue, extended linear mixing models have been proposed which lead to large scale nonsmooth ill-posed inverse problems. Furthermore, the regularization strategies used to obtain meaningful results have introduced interdependencies among abundance solutions that further increase the complexity of the resulting optimization problem. In this paper we present a novel data dependent multiscale model for hyperspectral unmixing accounting for spectral variability. The new method incorporates spatial contextual information to the abundances in extended linear mixing models by using a multiscale transform based on superpixels. The proposed method results in a fast algorithm that solves the abundance estimation problem only once in each scale during each iteration. Simulation results using synthetic and real images compare the performances, both in accuracy and execution time, of the proposed algorithm and other state-of-the-art solutions.

Introducing spatial prior information in hyperspectral imaging (HSI) analysis has led to an overall improvement of the performance of many HSI methods applied for denoising, classification, and unmixing. Extending such methodologies to nonlinear settings is not always straightforward, specially for unmixing problems where the consideration of spatial relationships between neighboring pixels might comprise intricate interactions between their fractional abundances and nonlinear contributions. In this paper, we consider a multiscale regularization strategy for nonlinear spectral unmixing with kernels. The proposed methodology splits the unmixing problem into two sub-problems at two different spatial scales: a coarse scale containing low-dimensional structures, and the original fine scale. The coarse spatial domain is defined using superpixels that result from a multiscale transformation. Spectral unmixing is then formulated as the solution of quadratically constrained optimization problems, which are solved efficiently by exploring their strong duality and a reformulation of their dual cost functions in the form of root-finding problems. Furthermore, we employ a theory-based statistical framework to devise a consistent strategy to estimate all required parameters, including both the regularization parameters of the algorithm and the number of superpixels of the transformation, resulting in a truly blind (from the parameters setting perspective) unmixing method. Experimental results attest the superior performance of the proposed method when comparing with other, state-of-the-art, related strategies.

Endmember (EM) variability has an important impact on the performance of hyperspectral image (HI) analysis algorithms. Recently, extended linear mixing models have been proposed to account for EM variability in the spectral unmixing (SU) problem. The direct use of these models has led to severely ill-posed optimization problems. Different regularization strategies have been considered to deal with this issue, but none so far has consistently exploited the information provided by the existence of multiple pure pixels often present in HIs. In this work, we propose to break the SU problem into a sequence of two problems. First, we use pure pixel information to estimate an interpolated tensor of scaling factors representing spectral variability. This is done by considering the spectral variability to be a smooth function over the HI and confining the energy of the scaling tensor to a low-rank structure. Afterwards, we solve a matrix-factorization problem to estimate the fractional abundances using the variability scaling factors estimated in the previous step, what leads to a significantly more well-posed problem. Simulations with synthetic and real data attest the effectiveness of the proposed strategy.

Sparse hyperspectral unmixing from large spectral libraries has been considered to circumvent the limitations of endmember extraction algorithms in many applications. This strategy often leads to ill-posed inverse problems, which can greatly benefit from spatial regularization strategies. However, existing spatial regularization strategies lead to large-scale nonsmooth optimization problems. Thus, efficiently introducing spatial context in the unmixing problem remains a challenge and a necessity for many real world applications. In this letter, a novel multiscale spatial regularization approach for sparse unmixing is proposed. The method uses a signal-adaptive spatial multiscale decomposition based on segmentation and oversegmentation algorithms to decompose the unmixing problem into two simpler problems: one in an approximation image domain and another in the original domain. Simulation results using both synthetic and real data indicate that the proposed method outperforms the state-of-the-art total variation-based algorithms with a computation time comparable to that of their unregularized counterparts.

In this paper, a new video super-resolution reconstruction (SRR) method with improved robustness to outliers is proposed. Although the regularized least mean squares (R-LMSs) are one of the SRR algorithms with the best reconstruction quality for its computational cost, and is naturally robust to registration inaccuracies, its performance is known to degrade severely in the presence of innovation outliers. By studying the proximal point cost function representation of the R-LMS iterative equation, a better understanding of its performance under different situations is attained. Using statistical properties of typical innovation outliers, a new cost function is then proposed and two new algorithms are derived, which present improved robustness to outliers while maintaining computational costs comparable with that of R-LMS. The Monte-Carlo simulation results illustrate that the proposed method outperforms the traditional and regularized versions of LMS, and is competitive with state-of-the-art SRR methods at a much smaller computational cost.

Endmember variability is an important factor for accurately unveiling vital information relating the pure materials and their distribution in hyperspectral images. Recently, the extended linear mixing model (ELMM) has been proposed as a modification of the linear mixing model (LMM) to consider endmember variability effects resulting mainly from illumination changes. In this paper, we further generalize the ELMM leading to a new model (GLMM) to account for more complex spectral distortions where different wavelength intervals can be affected unevenly. We also extend the existing methodology to jointly estimate the variability and the abundances for the GLMM. Simulations with real and synthetic data show that the unmixing process can benefit from the extra flexibility introduced by the GLMM.

This paper proposes a new kernel Kalman filter formulation for system identification and time series estimation in nonlinear time-varying environments. The unknown nonlinear time-varying function is approximated by a finite-order linear model in a reproducing kernel Hilbert space. The model coefficients define the state of the Kalman filter. Simulation results illustrate the improvement in estimation performance provided by the new algorithm when compared to the classical kernel LMS filter.

Kernel-based nonlinear mixing models have been applied to unmix spectral information of hyperspectral images when the type of mixing occurring in the scene is too complex or unknown. Such methods, however, usually require the inversion of matrices of sizes equal to the number of spectral bands. Reducing the computational load of these methods remains a challenge in large-scale applications. This paper proposes a centralized band selection (BS) method for supervised unmixing in the reproducing kernel Hilbert space. It is based upon the coherence criterion, which sets the largest value allowed for correlations between the basis kernel functions characterizing the selected bands in the unmixing model. We show that the proposed BS approach is equivalent to solving a maximum clique problem, i.e., searching for the biggest complete subgraph in a graph. Furthermore, we devise a strategy for selecting the coherence threshold and the Gaussian kernel bandwidth using coherence bounds for linearly independent bases. Simulation results illustrate the efficiency of the proposed method.

Mixing phenomena in hyperspectral images depend on a variety of factors, such as the resolution of observation devices, the properties of materials, and how these materials interact with incident light in the scene. Different parametric and nonparametric models have been considered to address hyperspectral unmixing problems. The simplest one is the linear mixing model. Nevertheless, it has been recognized that the mixing phenomena can also be nonlinear. The corresponding nonlinear analysis techniques are necessarily more challenging and complex than those employed for linear unmixing. Within this context, it makes sense to detect the nonlinearly mixed pixels in an image prior to its analysis, and then employ the simplest possible unmixing technique to analyze each pixel. In this paper, we propose a technique for detecting nonlinearly mixed pixels. The detection approach is based on the comparison of the reconstruction errors using both a Gaussian process regression model and a linear regression model. The two errors are combined into a detection statistics for which a probability density function can be reasonably approximated. We also propose an iterative endmember extraction algorithm to be employed in combination with the detection algorithm. The proposed detect-then-unmix strategy, which consists of extracting endmembers, detecting nonlinearly mixed pixels and unmixing, is tested with synthetic and real images.

Complex kernel-based adaptive algorithms have been recently introduced for complex-valued nonlinear system identification. These algorithms are built upon the same framework as complex linear adaptive filtering techniques and Wirtinger's calculus in complex reproducing kernel Hilbert spaces. In this paper, we study the convergence behavior of the augmented complex Gaussian KLMS algorithm. Simulation results illustrate the accuracy of the analysis.

The kernel least-mean-square (KLMS) algorithm is an appealing tool for online identification of nonlinear systems due to its simplicity and robustness. In addition to choosing a reproducing kernel and setting filter parameters, designing a KLMS adaptive filter requires to select a so-called dictionary in order to get a finite-order model. This dictionary has a significant impact on performance, and requires careful consideration. Theoretical analysis of KLMS as a function of dictionary setting has rarely, if ever, been addressed in the literature. In an analysis previously published by the authors, the dictionary elements were assumed to be governed by the same probability density function of the input data. In this paper, we modify this study by considering the dictionary as part of the filter parameters to be set. This theoretical analysis paves the way for future investigations on KLMS dictionary design.

This paper investigates the use of Gaussian processes to detect non-linearly mixed pixels in hyperspectral images. The proposed technique is independent of nonlinear mixing mechanism, and therefore is not restricted to any prescribed nonlinear mixing model. The observed reflectances are estimated using both the least squares method and a Gaussian process. The fitting errors of the two approaches are combined in a test statistics for which it is possible to estimate a detection threshold given a required probability of false alarm. The proposed detector is compared to a robust nonlinearity detector recently proposed using synthetic data and is shown to provide a better detection performance. The new detector is also tested on a real hyperspectral image.

Beamformer-assisted acoustic echo cancelers have raised a lot of interest lately. The same performance can be obtained with a reduced length acoustic echo canceler (AEC) as the beamformer (BF) performs spatial cancellation. Structures that jointly optimize the BF and the AEC coefficients are preferred in order to exploit synergies. Analytical models have been already derived for the behavior of the direct form implementation of such systems adapted using the constrained least-mean square (CLMS) algorithm. This work extends the analysis to the popular generalized sidelobe canceler (GSC) structure, while allowing for a positive definite step-size matrix. Analytical models are derived for the mean and mean-square behaviors of the adaptive coefficients. Simulation results are shown to be in excellent agreement with the performance predicted by the theory.

The Non-Negative Least-Mean-Square (NNLMS) algorithm and its variants have been proposed for online estimation under non-negativity constraints. The transient behavior of the NNLMS, Normalized NNLMS, Exponential NNLMS and Sign-Sign NNLMS algorithms have been studied in the literature. In this letter, we derive closed-form expressions for the steady-state excess mean-square error (EMSE) for the four algorithms. Simulation results illustrate the accuracy of the theoretical results. This work complements the understanding of the behavior of these algorithms.

This paper studies a pixel by pixel nonlinearity detector for hyperspectral image analysis. The reflectances of linearly mixed pixels are assumed to be a linear combination of known pure spectral components (endmembers) contaminated by additive white Gaussian noise. Nonlinear mixing, however, is not restricted to any prescribed nonlinear mixing model. The mixing coefficients (abundances) satisfy the physically motivated sum-to-one and positivity constraints. The proposed detection strategy considers the distance between an observed pixel and the hyperplane spanned by the endmembers to decide whether that pixel satisfies the linear mixing model (null hypothesis) or results from a more general nonlinear mixture (alternative hypothesis). The distribution of this distance is derived under the two hypotheses. Closed-form expressions are then obtained for the probabilities of false alarm and detection as functions of the test threshold. The proposed detector is compared to another nonlinearity detector recently investigated in the literature through simulations using synthetic data. It is also applied to a real hyperspectral image.

This work presents a statistical analysis of a beamformer-assisted acoustic echo canceler (AEC). A new formulation leads to analytical models that can also be used to predict the transient performance of adaptive wideband beamformers. Monte Carlo simulations illustrate the accuracy of the model, which is then used to provide design guidelines. Application of the new model confirms previous experimental findings that the same cancellation performance of a single-microphone AEC can be achieved with a shorter AEC when the possibility of spatial filtering is available.

Acoustic feedback is one of the main problems in modern hearing aids. It distorts the desired signal and limits the maximum stable gain. Results on acoustic feedback cancellation systems based on the prediction error method of closed-loop identification indicate that they perform better than most alternative solutions. Most available analyses of such systems, however, are limited to steady-state results. This paper presents a transient mean-square analysis of a recently proposed system. The structure is analyzed for slow adaptation and for autoregressive input signals. Analytical models are derived for the mean and mean-square adaptive weight behaviors. This includes a model for the transient behavior of the bias in the feedback path estimator. Monte Carlo simulations are presented to verify the accuracy of the derived models.

Like its linear counterpart, the Kernel Least Mean Square (KLMS) algorithm is also becoming popular in nonlinear adaptive filtering due to its simplicity and robustness. The "kernelization" of the linear adaptive filters modifies the statistics of the input signals, which now depends on the parameters of the used kernel. A Gaussian KLMS has two design parameters; the step size and the kernel bandwidth. Thus, new analytical models are required to predict the kernel-based algorithm behavior as a function of the design parameters. This pa per studies the stochastic behavior of the Gaussian KLMS algorithm for white Gaussian input signals. The resulting model accurately predicts the algorithm behavior and can be used for choosing the algorithm parameters in order to achieve a prescribed performance.

This paper proposes a new wavelet-packet transform based adaptive algorithm for sparse response identification. The distinctive features of the new algorithm are the on-line adaptability of the discrete wavelet packet transform (DWPT) and an efficient weight deactivation/activation schedule. The new algorithm, called mimetic wavelet-packet based (MWPB) algorithm, generalizes the WPB algorithm presented in. The MWPB algorithm presents better estimation and tracking performances than existing wavelet-based sparse response identification algorithms. Monte Carlo (MC) simulation results compare the performances of the four most recently proposed algorithms in this class.

In this paper, we propose a distributed learning strategy in wireless sensor networks. Taking advantage of recent developments on kernel-based machine learning, we consider a new sparsification criterion for online learning. As opposed to previously derived criteria, it is based on the estimated error and is therefore is well suited for tracking the evolution of systems over time. We also derive a gradient descent algorithm, and we demonstrate its relevance to estimate the dynamic evolution of temperature in a given region.

The process of adding high-density information onto printed material enables and improves interesting hardcopy document applications, such as: security, authentication, physical-electronic round tripping, item-level tagging as well as consumer/product interaction. This investigation on robust and high capacity print codes aims to maximize information payload in a given printed page area, subject to robustness to distortions originated by printing and scanning processes and also to degradations introduced by user manipulation of printed documents. The novel approach includes statistical print-and-scan channel characterization, designing of robust segmentation, unsupervised Bayesian color classification with expectation-maximization algorithm for parameters estimation of a mixture of Gaussians model and design of error correction codes. Results illustrate the performance evaluated under real channel and distortions conditions. High payload is achieved with sufficient robustness to distortions resulting of regular office hardcopy document handling: print-and-scan channel and user manipulation.

Adding high-density information to printed materials enables and improves interesting hardcopy document applications involving security, authentication, physical-electronic round tripping, item-level tagging, and consumer/product interaction. This investigation of robust and high capacity print codes aims to maximize information payload in a given printed page area, subject to robustness to channel errors including distortions introduced by the printing and scanning processes and also due to the usual degradations introduced by user manipulation of printed documents. The novel approach includes statistical print-and-scan channel characterization, designing of robust segmentation using visual cues, unsupervised Bayesian color classification with expectation-maximization algorithm for parameters estimation of a mixture of Gaussians model and design of error correction codes. Results illustrate the performance evaluated under real channel and distortions conditions. High payload is achieved with sufficient robustness to distortions resulting of regular office hardcopy document handling: print-and-scan channel and user manipulation.

This paper studies the statistical behavior of an affine combination of the outputs of two LMS adaptive filters that simultaneously adapt using the same white Gaussian input. The purpose of the combination is to obtain an LMS adaptive filter with fast convergence and small steady-state mean-square error (MSE). The linear combination studied is a generalization of the convex combination, in which the combination factor is restricted to the interval (0,1). The viewpoint is taken that each of the two filters produces dependent estimates of the unknown channel. Thus, there exists a sequence of optimal affine combining coefficients which minimizes the MSE. The optimal unrealizable affine combiner is studied and provides the best possible performance for this class. Then, a new scheme is proposed for practical applications. It is shown that the practical scheme yields close-to-optimal performance when properly designed (as suggested by the theoretical optimal).

This paper presents a novel implementation for identifying sparse telephone network echo channels. The new scheme follows the approach used in [1] in that the location of the channel response peak is estimated in the wavelet domain. A short time-domain adaptive filter is then located about the estimated peak to identify the sparse response. The primary purpose of this paper is to present an efficient design of such system. The use of a new block wavelet transform results in both 70% less computational complexity and improved peak detection. A new robust time-domain adaptive filtering is also proposed which significantly reduces the jitter problem in [1]. Monte Carlo simulations show excellent echo cancellation for a typical ITU-T channel.

This paper studies the behavior of the LMS algorithm for a special system identification problem when partial wavelet transformations restrict the algorithm's input vector to a subspace of the unknown system's input vector space. It is shown that the independence theory is not applicable in this case. A new theoretical model for the weight mean and fluctuation behaviors is developed which incorporates the correlation between successive data vectors (as opposed to using the independence theory model). Comparison of the new model predictions with Monte Carlo simulations shows good-to-excellent agreement, certainly much better than predicted by the independence theory model.

The super-resolution reconstruction (SRR) of images is an ill posed problem. Traditionally it is treated as a regularized minimization problem. Moreover, one of the major problems concerning SRR is its dependence on an accurate registration. In this work we show that a certain amount of registration error may. in fact, be beneficial for the performance of the least mean square SRR (LMS-SRR) adaptive algorithm. In these cases, the regularization term can be avoided and computational cost is reduced, an important advantage in real-time SRR applications.

This paper proposes improvements to the code division modulation and multiplexing (CDM) spread spectrum watermarking. The improved CDM technique exploits all the available information at the embedded and determines the required energy for the spreading sequences to achieve a specified bit error probability. This approach considers an additive noise in the channel and all the interferences from the host signal, cross-correlation between spreading sequences, perceptual masking and interferences from collusion attacks. We provide results to illustrate the improvements achieved and also comparisons with traditional CDM watermarking.

This paper proposes a wavelet-packet-based (WPB) algorithm for efficient identification of sparse impulse responses with arbitrary frequency spectra. The discrete wavelet packet transform (DWPT) is adaptively tailored to the energy distribution of the unknown system's response spectrum. The new algorithm leads to a reduced number of active coefficients and to a reduced computational complexity, when compared to competing wavelet-based algorithms. Simulation results illustrate the applicability of the proposed algorithm.

This paper provides new insights into on-line nonlinear sparse approximation of functions based on the coherence criterion. We revisit previous work, and propose tighter bounds on the approximation error based on the coherence criterion. Moreover, we study the connections between the coherence criterion and both the approximate linear dependence criterion and the principal component analysis. Finally, we derive a kernel normalized LMS algorithm based on the coherence criterion, which has linear computational complexity on the model order. Initial experimental results are presented on the performance of the algorithm.

This work presents a modified version of the variable step size Kwong and Johnston's algorithm (VSS) for LMS adaptive filtering. The new proposal, called robust variable step size (RVSS), presents less sensitivity to the power of the measurement noise with only a very small increase in the computational complexity. A theoretical analysis demonstrates the main properties of the new algorithm. For white Gaussian input signals the RVSS presents the same performance than the original VSS in a noise free environment. Simulation results are provided, showing the better performance of the new algorithm. The RVSS should find application, for example, in telephony applications when double talking interferences are significant

Super-resolution reconstruction of image sequences is highly dependent on the quality of the motion estimation between successive frames. This work presents a statistical analysis of the least mean square (LMS) algorithm applied to super-resolution reconstruction of an image sequence. Deterministic recursions are derived for the mean and mean square behaviors of the reconstruction error as functions of the registration errors. The new model describes the behavior of the algorithm in realistic situations, and significantly improves the accuracy of a simple model available in the literature. Monte Carlo simulations show good agreement between actual and predicted behaviors

We propose a new model for the behavior of the Least Mean Square (LMS) algorithm when implemented in finite precision. We model the adaptive filter coefficients as a Markov chain and determine its transition probability matrix for the one-dimensional case. We also determine conditions to avoid the so-called stopping phenomenon. The proposed model eliminates the linearizations used in previous models, accounts for saturation effects and leads to accurate estimations of the mean-square error behavior. Monte Carlo simulation results illustrate the quality of the proposed model.

Super-resolution reconstruction of image sequences is highly dependent on the data outliers and on the quality of the motion estimation. This work addresses the design of the least mean square (LMS) algorithm applied to super-resolution reconstruction. Based on a statistical model of the algorithm behavior, we propose a design strategy to reduce the effects of outliers on the reconstructed image sequence. We show that the proposed strategy can improve the algorithm performance in both transient and steady-state phases of adaptation in practical situations, when registration errors are considered.

This paper presents a new analytical model for the normalized least mean square (NLMS) adaptive algorithm. The new model is derived using a stochastic differential equation (SDE) approach. An accurate estimate of the steady-state weight-error correlations is also derived, which leads to an improved model performance for medium and large step sizes. Numerical simulations compare the new model with existing models and show better agreement with Monte Carlo simulations.

We show that the least-mean fourth (LMF) and the least-mean mixed-norm (LMMN) algorithms are not mean-square stable when the input regressor is Gaussian-distributed. For the LMF algorithm, we propose an upper bound for the algorithm's probability of divergence, given the input and noise statistics, the stepsize and the filter length. We show that the upper bound can also be used for the LMMN algorithm, which is a combination of LMS and LMF.

This paper proposes an informed spread spectrum watermarking technique for the embedding of multi-bits into images using blind detection. The new technique exploits the information available at the embedder to eliminate all the interferences from the host, patterns and perceptual masking. A statistical analysis leads to the minimum embedding strength required by each pattern to achieve a specified bit error probability for a given additive noise distortion. As a result, each message bit will present the same degree of robustness. Experimental results illustrate the performance achieved for a set of typical valumetric attacks.

The paper presents an analytical model for predicting the stochastic behavior of the affine projection (AP) algorithm operating in a nonstationary environment. The model is derived for autoregressive (AR) Gaussian inputs and for unity step size (fastest convergence). Deterministic recursive equations are presented for the mean weight and mean square error for a large number of adaptive taps, N, as compared to the algorithm order, P. The model predictions show excellent agreement with Monte Carlo simulations in transient and steady-state. The learning behavior of the AP algorithm in nonstationary environments is of great interest in applications such as acoustic echo cancellation.

The paper presents an analysis of the kurtosis performance surface as applied to linear estimation. The analysis concentrates on a modified kurtosis (MK) function used in implementations of the least mean kurtosis (LMK) adaptive algorithm. The MK function is shown to make the LMK algorithm applicable even for Gaussian inputs. The minimum of the MK function is derived and shown to be unique and to correspond to the Wiener solution of the mean square error (MSE) estimation problem. A quantitative comparison of the MSE and MK functions explains why the LMK adaptive algorithm is faster than MSE-based algorithms during the initial learning phase, becoming slower as it approaches steady-state.

This paper proposes a new recursive least squares adaptive algorithm, called the variable memory length (VML) algorithm. The new algorithm is robust in system identification problems in which the input power can be significantly reduced during operation. Most RLS-type algorithms tend to increase the error in the estimated weight vector in such situations. The VML algorithm keeps the mean square deviation of the weight unchanged during the absence of signal power. It should encounter application in systems such as automotive suspension fault detection and system identification using speech signals. In both cases, considerable periods of low input power during operation are common.

The paper presents a statistical analysis of the multi-split LMS algorithm. Deterministic recursions are obtained for the mean weight vector and the mean square error. Simulation results display excellent agreement with the theoretical predictions, and enable us to validate the proposed models for both transient and steady-state behaviors.

The paper presents an analytical model for predicting the stochastic behavior of the affine projection (AP) algorithm. The model is derived for autoregressive (AR) Gaussian inputs and for unity step size (fastest convergence). Deterministic recursive equations are presented for the mean weight and mean square error for a large number of adaptive taps, N, as compared to the algorithm order, P. The model predictions show better agreement between theory and simulations in transient and steady-state than previous models described in the literature. The learning behavior of the AP algorithm is of great interest in applications such as acoustic echo cancellation.

This paper presents a new analysis for the convergence of the LMF (least mean fourth) adaptive algorithm. The analysis improves previous results because it explicitly shows how the stability of the algorithm depends on the initial conditions of the weights, i.e., the analysis is also valid when the algorithm is initialized far from the optimum weight vector. Analytical expressions are derived relating the limiting values of the adaptation constant and the initial weight error vector. The analysis assumes a white zero-mean Gaussian reference signal and a white measurement noise with any even probability density function (p.d.f.) and finds conditions for convergence in the mean square sense.

This work studies the Minimum Output Variance Least Mean Square Estimator (MOV-LMS) when the output of the adaptive filter is constrained by a saturation-type nonlinearity. This situation is typical in several adaptive modeling and control systems where the associated hardware and transducers have a finite power handling capability. Analytical expressions are obtained for the behaviors of the mean weight vector and for mean square error for Gaussian inputs and slow learning. The optimum leakage factor is determined, which provides an unbiased solution to the associated nonlinear mean square estimation. problem. Monte Carlo simulations show excellent agreement between model and simulations for transient and steady-state conditions. The results in this paper demonstrate that the MOV-LMS algorithm with the optimized leakage factor has a superior steady-state performance than the LMS algorithm in the studied nonlinear environment.

This work presents a new analytical model for the Normalized Least Mean Square (NLMS) algorithm for Gaussian signals and large number of coefficients. These characteristics are of special interest in applications such as echo canceling and active noise control. The new results are compared to previous models described in the literature, showing a better agreement with Monte Carlo simulations even for unit step-size (maximum convergence speed). The new results contribute to an ongoing discussion about the relative performances of the NLMS and LMS algorithms. The new model shows that both algorithms have similar performances for a large number of coefficients and properly chosen step-sizes.

This work considers the practical situation where adaptive systems are subject to a saturation nonlinearity at the output of the adaptive filter. Such is the case in active control of noise and vibration. A new adaptive algorithm is proposed which implements the true stochastic gradient approach to the nonlinear problem. Deterministic nonlinear recursions are derived which model the mean weight and mean square error behaviors. The steady-state behavior is also studied. The practical aspects of nonlinearity estimation and hardware implementation are addressed. It is shown that the new algorithm outperforms the LMS algorithm even for considerable errors in estimating the nonlinearity parameters.

This paper presents a new methodology for evaluation and design of variable step size adaptive algorithms. The new methodology is based on a learning plane, which combines the evolutions of both the step size and the mean square error. It includes both transient and steady-state behaviors and can be used to compare performances of different algorithms against an optimum trajectory in the learning plane. The new technique can also be used for algorithm optimization in system identification applications.

This paper investigates the properties of the performance surface for a problem of nonlinear mean-square estimation of a random sequence. The problem studied has direct application to the study of active noise control (ANC) systems when the transducers are driven into nonlinear behavior. A deterministic expression is derived for the mean-square error (MSE) surface as a function of the system's degree of nonlinearity. It is shown how the presence of the nonlinearity deforms the MSE surface. It is demonstrated that the surface is unimodal, and the expression for the optimum weight vector is determined. The new results are then used to quantify the behavior of ANC systems employing the LMS adaptive algorithm. Important algorithm properties are derived from this study. Examples are presented to verify the analytical models derived.

This paper presents a stochastic analysis of the delayed least mean square (DLMS) adaptive algorithm using a new model. The new model does not use independence theory. Recursive difference equations are derived for the weight vector first and second moments. These equations yield new analytical results for the mean square error behavior. These results are compared to those of previous models. The new model is shown to be more general. The algorithm's properties are explained that could not be explained using existing models. The theoretical behavior is in close agreement with Monte Carlo simulations for the cases studied. This provides support for the accuracy of the theoretical model.

This paper presents a statistical analysis of the multiple-error Filtered-X Least Mean Square algorithm with nonlinearities in the secondary paths. Each nonlinearity is modelled by a scaled error function that can represent the saturation behavior of the transducers and associated hardware in active noise control applications. Deterministic recursions are derived for the mean weight and mean square error behavior for white Gaussian inputs and slow adaptation. The independence theory is not used in the derivations. Monte Carlo simulations show excellent agreement with the behavior predicted by the theoretical models. The new results predict the behavior of the algorithm for different amounts of nonlinear distortion.

This paper presents a statistical analysis of the least mean square (LMS) algorithm when a zero-memory nonlinearity appears at the adaptive filter output. The nonlinearity is modelled by a scaled error function. Deterministic nonlinear recursions are derived for the mean weight and mean square error (MSE) behavior for white Gaussian inputs and slow adaptation. Monte Carlo simulations show excellent agreement with the behavior predicted by the theoretical models. The analytical results show that a small nonlinear effect has a significant impact on the converged MSE.

This paper presents a new analytical model for the second moment behavior of the filtered-X LMS algorithm. The new model is not based on the independence theory, and is derived for Gaussian inputs and slow adaptation. Monte Carlo simulations show excellent agreement with the behavior predicted by the theoretical model.

This paper presents a stochastic analysis of the Filtered-X LMS algorithm. The mean weight vector recursion is derived for slow adaptation and for a white reference signal without use of independence theory. The Wiener solution is determined explicitly as a function of the input statistics and the impulse responses of the primary and secondary signal paths. It is shown that the steady-state mean weights for the Filtered-X LMS algorithm converge to the Wiener solution only if the estimate of the secondary path is without error. Monte Carlo simulations show excellent agreement with the behavior predicted by the theoretical model.

The least mean squares (LMS) algorithm is one of the most popular algorithms for digital implementation of real-time high-speed adaptive filters. This paper presents a study of the quantization effects in the finite precision LMS algorithm with power-of-two step sizes. Nonlinear recursions are derived for the mean and second moment matrix of the weight vector about the Wiener weight for white Gaussian data models and small algorithm step size /spl mu/. The solutions of these recursions are shown to agree very closely with the Monte Carlo simulations during all phases of the adaptation process. A design curve is presented to demonstrate the use of the theory to select the number of quantizer bits and the adaptation step size /spl mu/ to yield desired transient and steady-state behaviors.