It is demonstrated that the sampling distributions of the maximum likelihood (ML) estimator and its Studentized statistic for the generalized Gaussian distribution do not pass the most powerful normality tests even for fairly large sample sizes. This disagreement with what the standard large sample ML theory predicts and the computational burden of having to deal with its associated polygamma functions motivate the consideration of a competing convexity-based estimator. The asymptotic normality of this estimator is derived. It is shown that the competing estimator is almost as efficient as the ML estimator and its asymptotic relative efficiency to the ML estimator is equal to 1 in the limit as the shape parameter approaches zero. More important, its asymptotic distribution admits an exact variance stabilizing transformation, whereas the asymptotic variance function of the ML estimator does not have a closed-form variance stabilizing transformation. The exact transformation is a composition of the inverse hyperbolic cotangent and square root functions. Besides stabilizing the variance, the inverse hyperbolic cotangent and square root transformation is remarkably effective for symmetrizing and normalizing the sampling distribution of the estimator and hence improving the standard normal approximation. Furthermore, this simple transformation provides a quite accurate approximation to the non-closed-form variance stabilizing transformation of the ML estimator.

Many applications in real-time signal, image, and video processing require automatic algorithms for rapid characterizations of signals and images through fast estimation of their underlying statistical distributions. We present fast and globally convergent algorithms for estimating the three-parameter generalized gamma distribution (G Gamma D). The proposed method is based on novel scale-independent shape estimation (SISE) equations. We show that the SISE equations have a unique global root in their semi-infinite domains and the probability that the sample SISE equations have a unique global root tends to one. The consistency of the global root, its scale, and index shape estimators is obtained. Furthermore, we establish that, with probability tending to one, Newton-Raphson (NR) algorithms for solving the sample SISE equations converge globally to the unique root from any initial value in its given domain. In contrast to existing methods, another remarkable novelty is that the sample SISE equations are completely independent of gamma and polygamma functions and involve only elementary mathematical operations, making the algorithms well suited for real-time both hardware and software implementations. The SISE estimators also allow the maximum likelihood (ML) ratio procedure to be carried out for testing the generalized Gaussian distribution (GGD) versus the G Gamma D. Finally, the fast global convergence and accuracy of our algorithms for finite samples are demonstrated by both simulation studies and real image analysis.

Accurate estimation of the frequency of sinusoidal signals from noisy observations is an important problem in signal processing applications such as radar, sonar, and telecommunications. In this paper, we study the problem under the assumption of non-Gaussian noise in general and Laplace noise in particular. We prove that the Laplace maximum likelihood estimator is able to attain the asymptotic Cramer-Rao lower bound under the Laplace assumption which is one half of the Cramer-Rao lower bound in the Gaussian case. This provides the possibility of improving the currently most efficient methods such as nonlinear least-squares and periodogram maximization in non-Gaussian cases. We propose a computational procedure that overcomes the difficulty of local extrema in the likelihood function when computing the maximum likelihood estimator. We also provide some simulation results to validate the proposed approach.

Accurate estimation of sinusoidal frequencies from noisy observations is an important problem in many applications including radar, sonar, and data communications. Among many algorithms is the iterative filtering algorithm (IFA), proposed by Kay, which provides a computationally simple procedure yet capable of accurate frequency estimation especially at low signal-to-noise ratio (SNR). However, the convergence and other numerical/statistical properties of IFA have not been established beyond simulation. This paper makes several important contributions: (a) It shows that the poles of the AR filter must be reduced via a shrinkage parameter to accommodate possibly poor initial values. (b) It shows that the AR estimates in each iteration must be bias-corrected to produce more accurate frequency estimates; a closed-form expression is provided for bias correction. (c) It shows that for a sufficiently large sample size, the resulting algorithm, called new IFA, or NIFA, converges to the desired fixed-point which constitutes a consistent frequency estimator. Numerical examples, including a radar data example, are provided to demonstrate the findings.

We propose a novel methodology for estimating the shape parameter of a generalized Gaussian distribution (GGD). This new method is based on a simple estimating equation derived from a convex shape equation. The estimating equation is completely independent of gamma and polygamma functions. Thus, no lookup table, interpolation, or additional subroutine to evaluate these functions are required for real-time implementations of the proposed method, which is in contrast to all existing methods. Furthermore, we establish that the shape equation has a unique global root on the positive real line and the Newton-Raphson root-finding algorithm converges to the unique global root from any starting point in a semi-infinite interval /spl Theta//sub min/. More importantly, we show that the sample-based shape estimating equation has a unique global root with probability tending to one and the root is consistent for the true shape parameter. Finally, we prove via fixed point arguments that, with probability tending to one, the Newton-Raphson algorithm converges to the unique global root of the sample shape estimating equation from any starting point in /spl Theta//sub min/. Some numerical experiments are also provided to demonstrate the global convergence and the excellent finite sample performance of the proposed method.

Based on an asymptotic analysis of the contraction mapping (CM) method of Li and Kedem (1993), a bandwidth shrinkage rule is proposed for fast and accurate estimation of the frequencies of multiple sinusoids from noisy measurements. The CM frequency estimates are defined as the fixed points of a contractive mapping formed by the lag-one autocorrelation coefficient calculated from the output. of a parametric filter applied to the observed time series. With bandwidth parameters judiciously chosen according to the asymptotic analysis, the algorithm is shown to be able to accommodate possibly poor initial values of precision O(n/sup -1/3/) and converge to a final estimate whose accuracy is arbitrarily close to O(n/sup -3/2/), the optimal error rate for frequency estimation under the Gaussian assumption. The total computational complexity of the algorithm is shown to be O(n log n), which is comparable to that of n-point fast Fourier transform (FFT). A novelty in the asymptotic analysis is that it accommodates closely spaced frequencies by allowing not only the filter bandwidth but also the frequency separation to be functions of the sample size n. This enables an assessment of the accuracy of the frequency estimates for given bandwidths and initial values in situations where some or all of the frequencies are close to each other.

We present a general methodology for developing asymptotically distribution-free goodness-of-fit tests based on the Kullback-Leibler discrimination information. The tests are shown to be omnibus within an extremely large class of nonparametric global alternatives and to have good local power. The proposed test procedure is a nonparametric extension of the classical Neyman-Pearson log-likelihood ratio test and is based on mth-order spacings between order statistics cross-validated by the observed log likelihood. The developed method also generalizes Cox's procedure of testing separate families and covers virtually all parametric families of distributions encountered in statistics. It can also be viewed as a procedure based on sum-log functionals of nonparametric density-quantile estimators cross-validated by the log likelihood. With its good power properties, the method provides an extremely simple and potentially much better alternative to the classical empirical distribution function (EDF)-based test procedures. The important problem of selecting the order of spacings m in practice is also considered and a method based on maximizing the sample entropy constrained by the observed log likelihood is proposed. This data driven method of choosing m is demonstrated by Monte Carlo simulations to be more powerful than deterministic choices of m and thus provides a practically useful tool for implementing our test procedure.