The main premise of this article is to develop a maximum entropy estimation of an unknown distribution using order statistics. The exact solution using constraints on the order statistic means is derived. The result is an integral of a rational parametric function whose parameters depend on the order statistic constraints. This integral equation can be solved directly for a small number of order statistic means, e.g., 1, 2, or at most 3, via a multidimensional search in the parameter space. As the number of constraints increases, the search for the optimum parameters becomes formidable. To resolve this problem we have proposed an approximate solution to the resultant integral equation based on Bernstein polynomials and estimates of a desired number of quantiles from available samples. The proposed approximation approach does not require any search in the parameter space and is formulated for any number of samples from an unknown distribution. Performance of this approach is evaluated under various practical conditions by using Kullback-Leibler divergence function.

An analytical expression for a probability density is usually required in detection and estimation problems, yet it is usually only assumed or selected from contenders by parameter estimation, or the histogram is smoothed with an arbitrary window function. In contrast, given a histogram containing R sample points, we derive a nonlinear differential equation (NDEQ) whose solution is a maximum entropy density given constraints that arise from assumptions that the samples are means of the order statistics of the parent distribution. We solve the NDEQ for R=1 and approximate the solution for general R using the fact that order means partition the density into equal probability regions, which we require to independently be maximum entropy. Finally we show with a Rayleigh density example what errors may result.

This paper introduces a novel sparse Bayesian machine-learning algorithm for embedded feature selection in classification tasks. Our proposed algorithm, called the relevance sample feature machine (RSFM), is able to simultaneously choose the relevance samples and also the relevance features for regression or classification problems. We propose a separable model in feature and sample domains. Adopting a Bayesian approach and using Gaussian priors, the learned model by RSFM is sparse in both sample and feature domains. The proposed algorithm is an extension of the standard RVM algorithm, which only opts for sparsity in the sample domain. Experimental comparisons on synthetic as well as benchmark data sets show that RSFM is successful in both feature selection (eliminating the irrelevant features) and accurate classification. The main advantages of our proposed algorithm are: less system complexity, better generalization and avoiding overfitting, and less computational cost during the testing stage.

This paper presents a method for automatic earthquake signal phase picking based on the continuous wavelet transform (CWT) of the seismogram, namely, the wavelet phase picker (WPP). A characteristic function is defined using the envelope function of CWT coefficients of the vertical component seismogram. The performance of the WPP method is evaluated on a database consisting of more than 460 local seismograms, and the P-phases detected by the algorithm are compared with those reported in the database. This evaluation confirms the high accuracy of automatically detected picks for the majority of seismograms. In addition, the results obtained by the algorithm are compared with a well-known method of P-phase picking referred to as the autoregressive Akaike information criteria. This comparison demonstrates the reliability of the proposed method.

In the above paper (ibid., vol. 51, no. 5, pp. 2666-2674, May 2013), there is an error in equation (7) due to a typesetting error. The correct form of the equation is presented here.

The paper proposes an efficient method for very short-term prediction of wind turbine power output. The method, which models the turbine as a Hammerstein system, exploits an adaptive linear filtering algorithm. The performance of the proposed method is examined by implementation of two linear adaptive algorithms, namely, least mean squares (LMS) and recursive least squares (RLS) filters. Using synthetic generation of turbine power output, it is shown that the RLS algorithm gives more accurate results with moderate computational burden as compared to the LMS algorithm and rival artificial neural networks.

A classifier combining strategy, virtual voting by random projection (VVRP), is presented. VVRP takes advantage from the bounded distortion incurred by random projection in order to improve accuracies of stable classifiers like discriminant analysis (DA) where existing classifier combining strategies are known to be failed. It uses the distortion to virtually generate different training sets from the total available training samples in a way that does not have the potential for overfitting. Then, a majority voting combines the base learners trained on these versions of the original problem. VVRP is very simple and just needs determining a proper dimensionality for the versions, an often very easy task. It is shown to be stable in a very large region of the hyperplane constructed by the dimensionality and the number of the versions. VVRP improves the best state-of-the-art DA algorithms in both small and large sample size problems in various classification fields.

Extended Kalman filter (EKF) has previously been employed to extract road maps in satellite images. This filter traces a single road until a stopping criterion is satisfied. In our new approach, we have combined EKF with a special particle filter (PF) in order to regain the trace of the road beyond obstacles, as well as to find and follow different road branches after reaching to a road junction. In this approach, first, EKF traces a road until a stopping criterion is met. Then, instead of terminating the process, the results are passed to the PF algorithm which tries to find the continuation of the road after a possible obstacle or to identify all possible road branches that might exist on the other side of a road junction. For further improvement, we have modified the procedure for obtaining the measurements by decoupling this process from the current state prediction of the filter. Removing the dependence of the measurement data to the predicted state reduces the potential for instability of the road-tracing algorithm. Furthermore, we have constructed a method for dynamic clustering of the road profiles in order to maintain tracking when the road profile undergoes some variations due to changes in the road width and intensity.

A novel Bayesian-based speckle suppression method for Synthetic Aperture Radar ( SAR) images is presented that preserves the structural features and textural information of the scene. First, the logarithmic transform of the original image is analyzed into the multiscale wavelet domain. We show that the wavelet coefficients of SAR images have significantly non-Gaussian statistics that are best described by the 2-D GARCH model. By using the 2-D GARCH model on the wavelet coefficients, we are capable of taking into account important characteristics of wavelet coefficients, such as heavy tailed marginal distribution and the dependencies between the coefficients. Furthermore, we use a maximum a posteriori (MAP) estimator for estimating the clean image wavelet coefficients. Finally, we compare our proposed method with various speckle suppression methods applied on synthetic and actual SAR images and we verify the performance improvement in utilizing the new strategy.

In this work, we propose an efficient framework for compressing and displaying medical images. Image compression for medical applications, due to available Digital Imaging and Communications in Medicine requirements, is limited to the standard discrete cosine transform-based joint picture expert group. The objective of this work is to develop a set of quantization tables (Q tables) for compression of a specific class of medical image sequences, namely echocardiac. The main issue of concern is to achieve a Q table that matches the specific application and can linearly change the compression rate by adjusting the gain factor. This goal is achieved by considering the region of interest, optimum bit allocation, human visual system constraint, and optimum coding technique. These parameters are jointly optimized to design a Q table that works robustly for a category of medical images. Application of this approach to echocardiac images shows high subjective and quantitative performance. The proposed approach exhibits objectively a 2.16-dB improvement in the peak signal-to-noise ratio and subjectively 25% improvement over the most useable compression techniques.

A new nonlinear image filtering technique for multiplicative noise in the wavelet transform domain is presented. Unlike most existing techniques, this approach does not require prior modeling of either the image or the noise statistics. This approach explicitly accounts for the signal dependent nature of the noise. This is considered by attempting to use the variance of the detail wavelet coefficients to decide whether to smooth or preserve these coefficients. The algorithm uses the statistical approach of analysis of variance to decide whether detail wavelet coefficients are due to image related features or they are due to noise. Results of our approach show that the proposed algorithm outperforms other conventional techniques.

A new image denoising technique in the wavelet transform domain for multiplicative noise is presented. Unlike most existing techniques, this approach does not require prior modeling of either the image or the noise statistics. It uses the variance of the detail wavelet coefficients to decide whether to smooth or to preserve these coefficients. The approach takes advantage of wavelet transform property in generating three detail subimages each providing specific information with certain feature directivity. This allows the ability to combine information provided by different detail subimages to direct the filtering operation. The algorithm uses the hypothesis test based on the F-distribution to decide whether detail wavelet coefficients are due to image related features or they are due to noise. The effectiveness of the proposed technique is tested for orthogonal as well as biorthogonal mother wavelets in order to study the effect of the smoothing process under different wavelet types.

In this work, a new approach is proposed that deals with blocking effects in JPEG compressed images. High-frequency details of the coded images are mainly contaminated by quantization noise. Preserving the image details and reducing the effect of quantization noise as much as possible can improve the ability of any enhancing method. To achieve this goal along with the removal of the blocking effect, the high-frequency components of the image are first extracted by high pass filtering. The result is then scaled by a factor that depends on the compression ratio and subtracted from the observed image. This result is used to design an adaptive filter that depends on the statistical behavior of the preprocessed image. The adaptive filter is applied to the resultant image. The result shows high SNR, significant improvement in the separation between blocking noise and image features, and effective reduction of image blurring. Other steps are required to preserve the global and local edges of the processed image, remove blocking noise, and ensure smoothness without blurring. These steps are dedicated to remove blocking artifacts and to enhance feature regularities. The evaluation of this approach in comparison with other techniques is carried out both subjectively and qualitatively.

A new approach for image denoising in the wavelet transform domain is proposed. In this approach we attempt to replace each wavelet coefficient by its expected value. For that we will use local neighboring coefficients to provide a measure of similarity, noise and edge classification. The approach uses the statistical characteristics of neighboring coefficients as well as the noise characteristics. A clustering technique is used to determine the degree of belonging of neighboring coefficients and coefficient under consideration. Experimental results show that this technique yields comparable results in removing Gaussian type noise. The results show that the approach yields far better results than other existing technique in removing both Gaussian and outlier type noise without disturbing important image features.

A novel fuzzy filter is proposed. As recent studies show (Y. Choi and R. Krishnapuram, 1997), developing a rule based system capable of addressing all the different issues in filtering seems to be a tedious task and requires a large rule base. But nevertheless, if we can transform the features (such as amplitude, location and/or time index) into a higher dimension feature space, we can have a data set with more separation between "noisy" and "clean" signals. Therefore, the decision boundaries would be mathematically less complicated, and consequently a smaller number of rules is needed.

Nonlinear filtering has found many practical applications in digital signal and image processing. The computation complexity of these filtering algorithms make them difficult for real-time hardware implementation. One of these nonlinear filters, which is based on fuzzy classification of each pixel to subgroups of its neighboring pixels, is considered hen for hardware implementation. The criteria of this filter are based on the local context which form the basis of the fuzzy rule. The filtering algorithm is slightly modified for implementation into a Xilinx Virtex series of FPGA for real-time processing of image sequences. Implementation details and recommendations for further improvement are discussed. Result of a simulation example from the proposed hardware implementation is also presented.

Filtering noise in real-time image sequences is required in some applications like medical imaging. The optimum approach in this case is in the form of adaptive 3-D spatial-temporal filter, which is generally very complex and prohibitive for real-time implementation. Independent processing of the image sequences, in spatial and temporal domains can resolve some of these implementation difficulties. Some of the existing spatial filters can easily be modified for real-time implementation. Adaptive temporal filters, however, are more involved. In this paper, an adaptive temporal filter is proposed that lend itself to hardware implementation for real-time temporal processing of image sequences. The proposed algorithm is based on adaptive Kalman filtering which is relatively simple and effective in its performance. Adaptation in this case is with respect to motion in the image sequence as well as variation of noise statistics. An efficient hardware implementation of this algorithm, based on FPGA technology, is proposed.

A generalized nonlinear filter called the fuzzy cluster filter is introduced. This filter applies fuzzy clustering inside a running-window to estimate the clean output (i.e., geometrical center of the window). This filter is capable of cancelling the heavy-tailed contaminated Gaussian noise with a good performance. The results on some signals and images demonstrate the efficacy of this approach. The performance of the method is found by calculating the mean-square-error (MSE) for different signal-to-noise ratios using Monte Carlo simulations.

A new approach for high-resolution direction-of-arrival (DOA) estimation of coherent wideband signals is proposed. The approach is based on the projection of the two-dimensional spatial-temporal DFT of the received signal. The projections are used to construct narrowband covariance matrices having the same signal subspace which are then combined to yield the array covariance matrix. The array covariance can then be used by any signal-subspace eigendecomposition algorithm. The proposed approach does not require any preliminary estimates of DOAs and is not restricted to particular source models.>

A method for the suppression of coherent noise in seismic data based on the eigendecomposition of a data covariance matrix is demonstrated. Based on the Karhunen-Loeve transform, the proposed procedure is useful against noise energy exhibiting both two-dimensional space and time coherencies or coherent two-dimensional patterns which are not necessarily linear and therefore cannot generally be velocity-filtered. This method trains on a region containing the undesired coherent noise; the dominant eigenvectors determined from the covariance matrix of that noise are used to reconstruct the noise in the region of interest. Subtracting the reconstruction from the original data leaves a residual in which the coherent noise has been suppressed. In the example considered, this method effectively suppresses the noise in a record of marine seismic data containing backscattered source energy.>

An algorithm for automatic well-log signal segmentation previously proposed by the author (ibid., vol.27, p.633-41 (1987)) demands knowledge of the number of segments or another similar constraint as a stopping rule. An information-theoretic criterion is proposed to estimate the number of segments while the proposed sequential segmentation algorithm is in operation. A recursive procedure is developed to update the criterion function sequentially with minimum computation effort.>

A novel training approach based on the back-propagation algorithm is introduced. In the proposed approach, initially, a set of training vectors is obtained by applying the Karhunen-Loe've transform on the training patterns. The training is first started in the direction of the major eigenvectors of the correlation matrix of the training patterns and then continues by gradually including the remaining components, in their order of significance. With this approach, the number of computations is significantly reduced and the learning rate is improved. The performance of this method is compared with the standard back-propagation algorithm in segmenting a synthetic noisy image.

The author proposes a method, called spatial filtering, to resolve some of the problems associated with spatial smoothing. The method, which is similar to spatial smoothing, can be applied to wideband as well as narrowband problems. The objective of the method is to decorrelate completely the set of fully correlated sources. The theoretical findings are supported by simulations.>

Target tracking with Kalman filters is hampered by target maneuvering and unknown process and measurement noises. We show that moving data windows may be used to analyze state and measurement error sequences, determining robust estimates of bias and covariance. For unknown large steps in the system forcing functions and non-Gaussian measurement errors the robust estimators yield improvements over linear bias and covariance estimators. Extensive simulations compare conventional, linear adaptive and robust adaptive average step responses of a first-order system filter.