The all-pole model is often used to approximate rational transfer functions parsimoniously. In many applications, such as single channel blind deconvolution, an estimate of the channel is required. However, in general, attempting to model the entire channel spectrum by a single all-pole model leads to a large computational load. Hence, it is better to model a particular frequency band of the spectrum by an all-pole model, reducing a single high-dimensional optimisation to a number of low-dimensional ones. If each subband is completely decoupled from the others, and does not enforce any continuity, there are discontinuities in the spectrum at the subband boundaries. Continuity is ensured by constraining the subband parameters such that the end points at one subband boundary are matched to the spectrum in the adjacent subbands. This is formulated in the Bayesian probabilistic framework.

Blind deconvolution is fundamental in signal processing applications and, in particular, the single channel case remains a challenging and formidable problem. This paper considers single channel blind deconvolution in the case where the degraded observed signal may be modeled as the convolution of a nonstationary source signal with a stationary distortion operator. The important feature that the source is nonstationary while the channel is stationary facilitates the unambiguous identification of either the source or channel, and deconvolution is possible, whereas if the source and channel are both stationary, identification is ambiguous. The parameters for the channel are estimated by modeling the source as a time-varyng AR process and the distortion by an all-pole filter, and using the Bayesian framework for parameter estimation. This estimate can then be used to deconvolve the observed signal. In contrast to the classical histogram approach for estimating the channel poles, where the technique merely relies on the fact that the channel is actually stationary rather than modeling it as so, the proposed Bayesian method does take account for the channel's stationarity in the model and, consequently, is more robust. The properties of this model are investigated, and the advantage of utilizing the nonstationarity of a system rather than considering it as a curse is discussed.

This letter describes an efficient method to perform nonstationary signal classification. A support vector machine (SVM) algorithm is introduced and its parameters optimized in a principled way. Simulations demonstrate that our low-complexity method outperforms state-of-the-art nonstationary signal classification techniques.

In this paper we address the problem of on-line source separation with sources modelled as mixtures of Gaussians which are linearly combined via a series of non-stationary mixing matrices. The online recovery of the sources from the observations is a non-linear statistical filtering problem that we address using state of the art particle filter methods. Simulations are presented and satisfactory results are obtained.

Separability of signal mixtures given only one mixture observation is defined as the identification of the accuracy to which the signals can be separated. The paper shows that when signals are separated using the generalised Wiener filter, the degree of separability can be deduced from the filter structure. To identify this structure, the processes are represented on an arbitrary spectral domain, and a sufficient solution to the Wiener filter is obtained. The filter is composed of a term independent of the signal values, corresponding to regions in the spectral domain where the desired signal components are not distorted by interfering noise components, and a term dependent on the signal correlations, corresponding to the region where components overlap. An example of determining perfect separability of modulated random signals is given.

We present an unsupervised segmentation algorithm comprising an annealing process to select the maximum a posteriori (MAP) realization of a hierarchical Markov random field (MRF) model. The algorithm consists of a sampling framework which unifies the processes of model selection, parameter estimation and image segmentation, in a single Markov chain. To achieve this, reversible jumps are incorporated into the Markov chain to allow movement between model spaces. By using partial decoupling to segment the MRF it is possible to generate jump proposals efficiently while providing a mechanism for the use of deterministic methods, such as Gabor filtering, to speed up convergence.

Markov chain Monte Carlo (MCMC) samplers have been a very powerful methodology for estimating signal parameters. With the introduction of the reversible jump MCMC sampler, which is a Metropolis-Hastings method adapted to general state spaces, the potential of the MCMC methods has risen to a new level. Consequently, the MCMC methods currently play a major role in many research activities. In this paper we propose a reversible jump MCMC sampler based on predictive densities obtained by integrating out unwanted parameters. The proposal densities are approximations of the posterior distributions of the remaining parameters obtained by sampling importance resampling (SIR). We apply the method to the problem of signal detection and parameter estimation of signals. To illustrate the proposed procedure, we present an example of sinusoids embedded in noise.

A polynomial approximation to the likelihood function allows for marginalised estimates of model parameters to be obtained in the form of a Volterra series. The series can be applied directly to the observed data vector in an iterative fashion, to converge upon a set of parameter MAP estimates with low computational cost. A sample application towards OCR is used as an illustration.

Modeling and reconstruction methods are presented for noise reduction of autocorrelated signals in non-Gaussian, impulsive noise environments. A Bayesian probabilistic framework is adopted and Markov chain Monte Carlo methods are developed for detection and correction of impulses. Individual noise sources are modeled as Gaussian with unknown scale (variance), allowing for robustness to "heavy-tailed" impulse distributions, while the underlying signal is modeled as autoregressive (AR). Results are presented for both artificial and real data from voice and music recordings, and comparisons are made with existing techniques. The new techniques are found to give improved detection and elimination of impulses in adverse noise conditions at the expense of some extra computational complexity.

A set of approximations has been applied to allow the inclusion of Gaussian distributed priors for the linear parameters of the general linear model in order that the parameters may be integrated out alongside the Gaussian error noise variance, to give the model evidence and posterior distributions in analytic form. The extended model achieves greater accuracy in parameter estimation and evidence approximation when applied in a Bayesian inference framework, with no increase in computational load.

With the imminent widespread availability of digital video broadcasts and the subsequent increase in the demand for broadcast material, image sequence restoration is an increasing source of concern for both archivists and broadcasters. This paper presents a two stage technique for registering the lines in video data digitized from a noisy source. In such situations the horizontal synchronization pulses may not have the correct amplitude causing the loss of 'lock' in the digitizing apparatus. The effect is that the image lines are randomly shifted horizontally with respect to their true locations. This manifests as jagged vertical edges in the observed sequence, an annoying artefact. The algorithm presented here relies on a two-dimensional autoregressive (2D AR) model of the image to measure the line displacements using a multiresolution scheme.

For the estimation of the model coefficients of a polynomial autoregressive process with non-Gaussian innovations least l/sub p/-norm estimation (LLPN) is suggested. Simulations showed that LLPN estimation leads to better estimates than the least squares estimation in terms of the mean and the standard deviations of the estimates. The algorithm is also employed in modeling audio data in non-Gaussian noise with the objective of separating signal from noise and superior results have been obtained when compared to the linear autoregressive modeling. Directions of future research are also addressed.

Most of the existing coefficient estimation techniques in the literature for autoregressive (AR) symmetric /spl alpha/-stable (S/spl alpha/S) processes require large amounts of data for efficient estimation. However, in many practical cases, either only a short length of data is available or the data is nonstationary. Motivated by the norm of /spl alpha/-stable variables, the AR model coefficient estimation problem is formulated as an l/sub p/-norm minimization problem, and the interactively reweighted least squares (IRLS) is suggested for the solution. The simulation results indicate superior performance when compared to existing methods, especially when only short length data are available.

The separation of independent sources from mixed observed data is a fundamental and challenging problem. In many practical situations, observations may be modelled as linear mixtures of a number of source signals, i.e. a linear multi-input multi-output system. A typical example is speech recordings made in an acoustic environment in the presence of background noise and/or competing speakers. Other examples include EEG signals, passive sonar applications and cross-talk in data communications. We propose iterative algorithms to solve the n/spl times/n linear time invariant system under two different constraints. Some existing solutions for 2/spl times/2 systems are reviewed and compared.

Statistical model-based methods are presented for the reconstruction of autocorrelated signals in impulsive plus continuous noise environments. Signals are modelled as autoregressive and noise sources as discrete and continuous mixtures of Gaussians, allowing for robustness in highly impulsive and non-Gaussian environments. Markov Chain Monte Carlo methods are used for reconstruction of the corrupted waveforms within a Bayesian probabilistic framework and results are presented for contaminated voice and audio signals.

In this paper we derive the a posteriori probability for the location of bursts of noise additively superimposed on a Gaussian AR process. The theory is developed to give a sequentially based restoration algorithm suitable for real-time applications. The algorithm is particularly appropriate for digital audio restoration, where clicks and scratches may be modelled as additive bursts of noise. Experiments are carried out on both real audio data and synthetic AR processes and significant improvements are demonstrated over existing restoration techniques.>

Bright and dark flashes are typical artifacts in degraded motion picture material. The distortion is referred to as "dirt and sparkle" in the motion picture industry. This is caused either by dirt becoming attached to the frames of the film, or by the film material being abraded. The visual result is random patches of the frames having grey level values totally unrelated to the initial information at those sites. To restore the film without causing distortion to areas of the frames that are not affected, the locations of the blotches must be identified. Heuristic and model-based methods for the detection of these missing data regions are presented in this paper, and their action on simulated and real sequences is compared.

This paper presents a number of model based interpolation schemes tailored to the problem of interpolating missing regions in image sequences. These missing regions may be of arbitrary size and of random, but known, location. This problem occurs regularly with archived film material. The film is abraded or obscured in patches, giving rise to bright and dark flashes, known as "dirt and sparkle" in the motion picture industry. Both 3-D autoregressive models and 3-D Markov random fields are considered in the formulation of the different reconstruction processes. The models act along motion directions estimated using a multiresolution block matching scheme. It is possible to address this sort of impulsive noise suppression problem with median filters, and comparisons with earlier work using multilevel median filters are performed. These comparisons demonstrate the higher reconstruction fidelity of the new interpolators.

The ability of connectionist networks to generalize is often cited as one of their most important properties. We analyze the generalization ability of the class of generalized single-layer networks (GSLNs), which includes Volterra networks, radial basis function networks, regularization networks, and the modified Kanerva model, using techniques based on the theory of probably approximately correct (PAC) learning which have previously been used to analyze the generalization ability of feedforward networks of linear threshold elements (LTEs). An introduction to the relevant computational learning theory is included. We derive necessary and sufficient conditions on the number of training examples required by a GSLN to guarantee a particular generalization performance. We compare our results to those given previously for feedforward networks of LTEs and show that, on the basis of the currently available bounds, the sufficient number of training examples for GSLNs is typically considerably less than for feedforward networks of LTEs with the same number of weights. We show that the use of self-structuring techniques for GSLNs may reduce the number of training examples sufficient to guarantee good generalization performance, and we provide an explanation for the fact that GSLNs can require a relatively large number of weights.

Many real world systems can only be described well by non-linear models. The analysis and use of non-linear models can be very difficult and time consuming. An attractive class of models is one whose analysis can be based directly on linear systems analysis. One such class comprises models that are linear-in-the-parameters. Such models tend to have extremely large numbers of parameters, although only a handful may be relevant for any particular data set. An algorithm is described that enables the important parameters of a model to be found. Bayesian inference is used to give a consistent framework for fitting models and selecting between competing models. Simulations are given illustrating the problems of parameter estimation and model selection.>

Discusses model order selection in relation to the discrete Karhunen-Loeve transform (DKLT) and the singular value decomposition (SVD). There are many applications of the DKLT and SVD where it is necessary to discard some of the small singular values that may represent corrupted signal information. Bayesian methods allow to determine the DKLT model order evidence which indicates the optimal number of basis vectors to choose for reconstruction such that the signal is not over-parameterised. Evidence methods can also be used for the SVD to determine the number of singular values (and hence the effective rank) of a singular or ill-conditioned matrix,.>

Perhaps the most common distortion in degraded film or video material is the occurrence of blotches of varying colour randomly dispersed in each frame. The blotches are caused by the abrasion of the film or collection of dirt on the film as it passes through the projection apparatus. They manifest as flashes of bright and dark areas. The blotches represent regions of missing data (i.e. they replace the original data), and the paper discusses a model based technique for suppressing this artefact.>

The a posteriori probability for the location of bursts of noise additively superimposed on a Gaussian autoregressive (AR) process is derived. The maximum a posteriori (MAP) solution for noise burst position is obtained by using a simple search procedure, yielding the noise burst location corresponding to minimum probability of error. This procedure finds application in digital audio processing, where clicks and scratches may be modeled as additive bursts of noise. The method permits accurate detection of these degradations and their subsequent replacement (interpolation). Experiments were carried out on both real audio data and synthetic AR processes, and comparisons are made with previous techniques.>

The pattern classification and generalization ability of the class of generalized single-layer networks (GSLNs) using techniques from computational learning theory is analyzed. The authors derive necessary and sufficient conditions on the number of training examples required in order to guarantee a particular generalization performance and compare the bounds obtained to those available for (multilayer) feedforward networks of linear threshold elements (LTEs). This allows one to show that, on the basis of currently available bounds, the sufficient number of training examples for GSLNs tends to be considerably less than for feedforward networks of LTEs with the same number of weights. It is also shown that the use of self-structuring techniques for GSLNs may reduce the number of training examples sufficient for good generalization. An explanation for the fact that GSLNs can require a relatively large number of weights is given.>

The authors describe the development of a new form of connectionist model or neural network which, although having a unimodal error surface, will optimize its structure adaptively. The network is derived from the Volterra connectionist model. The adaptive noninformative extension nonlinearities removal method is used. The method requires only a slight computational expenditure in order to perform the self-structuring operation and may be operated with a bias to network contraction or reduction of error. Also, it does not require exact redundancies in order to operate. In addition, the inherent null-space nonlinearities removal method demonstrates a much more fundamental basis for removal of redundancy. This approach provides a deeper understanding of the problem and may in its current form be used as a block method with training set data.>

The authors examine the problem of estimating signals corrupted by additive non-Gaussian noise. Since the linear filter is known to be optimal if the noise is Gaussian, they apply general nonlinear filters, based on Volterra series, to the non-Gaussian case. Nonlinear Wiener filters are derived, and their performance investigated in example non-Gaussian noise densities.>

The output mapping method is used to demonstrate an approach which makes the Volterra connectionist model highly efficient computationally, in comparison with current neural networks. Although the system was applied to yield linear recognizers, exactly the same approach can be used to reduce the order of a nonlinear recognizer. The network is not constrained to fit arbitrary indices and may utilize all of its degrees of freedom to improve recognition performance.>

A stroke segmentation algorithm based on curvature information processing and scale-space filtering is proposed. The strokes extracted are very close to strokes described by psychophysicists. A simple online author-dependent recognition system that uses the segmentation algorithm as a preprocessing stage is described and its performance evaluated for handwritten data obtained from five authors. The segmentation results indicate a performance much better than that of techniques based on direct estimation of penspeed or curvature. The scale-space plot of stroke boundaries also locates many descriptive features detectable in cursive script, allowing the programmer to choose any desired scale or scales. Once segmentation is performed, the problem of recognizing recursive script is not much more difficult than the problem of recognizing isolated characters. The performance of the recognition system appears to be better than that of any cursive script recognition system designed so far, even though extremely simplifying assumptions have been made in the stroke matching stage.>

A connectionist model that is introduced based on the nonlinear extension of adaptive filter theory is introduced. It is shown that the model converges in the learning process to a global optimum. Experimental results indicate that the rate of convergence is considerably faster than has been reported for other models. It is concluded that the extended space approach leads to networks that can, with sufficient extension, synthesize any nonlinear discriminant function while maintaining unimodality. The adaptive filter knowledge also allows analytical solutions to the vital problem of parameter tuning, thus allowing good first-run performance on practical data.>

The degrading effects of nonstationary and abruptly changing signal characteristics on the performance of a particular class of restoration systems, namely those which use two sources of information, are discussed. The system considered is the adaptive noise suppression system. In this system abrupt changes in signal statistics result in transient degradations in the form of excessive suppression of some signal frequency components and or inadequate suppression of noise frequency components. Abrupt changes are detected using an appropriate distance between two autoregressive models of the signal. After detection of an abrupt change the adaptation gain is increased and/or the filter parameters are reinitialized in order to minimize the degradation.>

A method is presented for the adaptive registration of two images between which the image object is shifted. The method is based on multidimensional least-mean-squares (LMS) adaptive filters. The filters are derived, and the performance surface is shown to be unimodal. The multidimensional filter's similarities to and differences from the one-dimensional LMS adaptive filter are briefly discussed. It is shown that the filter will perform the necessary operations of correlation, interpolation, filtering, and shifts to produce a registered output image. An algorithm is developed to work in conjunction with the filter to provide explicit shift measurements. The advantages of this inherently parallel adaptive approach over current methods are discussed and shown to be most apparent when the images are corrupted by noise or are related by spatially varying shifts. The possibility of application of multidimensional LMS adaptive filters to pattern recognition is mentioned.>

An FDM/TDM (frequency-division multiplexed/time-division multiplexed) transmultiplexer may be implemented as a periodically time-varying (PTV) infinite-impulse response (IIR) filter. It is argued that the IIR filter should provide higher efficiency in terms of the number of multiplications required than the FIR (finite-impulse response) filter bank structure typically used for the FDM/TDM transmultiplexer. An attempt is made to apply adaptive filter theory to solve this design problem. Using this approach, it has become possible to design transmultiplexer systems with up to 12 channels, which is more than by previous numerical methods.>

The authors present a method for restoration of archived gramophone recordings for the case in which more than one copy of a record is available. Often, played-back signals are not in synchronism. Synchronization of two sampled waveforms usually requires that one of the signals be interpolated. An adaptive FIR (finite-impulse response) interpolator based on the least-square error criterion is used to synchronize the signals. Analysis of the interpolator shows that in addition to interpolation significant noise rejection is achieved at its output. Since least-squares filters are sensitive to outliers it is necessary to filter out relatively large impulsive disturbances prior to adaptive filtering. The authors present a detection-estimation algorithm for impulsive noise removal.>

A Mixed Integer Linear Programming (MILP) technique for the optimal design of windowed FFTs with discrete coefficients is presented. The technique uses a frequency domain optimisation suitable for spectrum analysis applications. The complex coefficients are constrained to have real and imaginary components which are integer powers of two or the sum of two integer powers of two. This results in a multiplierless system with significant savings in time and hardware costs.

A new algorithm is proposed for the digital implementation of theM-channel SSB-frequency-division multiplexing (FDM) modulator and demodulator, which is computationally attractive and inherently insensitive to parameter quantization.