In this letter we present a framework for estimating sparse covariance matrices, wherein we solve the$\ell _{0}-$norm penalized maximum likelihood estimation problem using the extended Bayesian information criterion (EBIC), a high dimensional model selection rule. The framework combines choosing the sparsity pattern and estimating the covariance matrix in a single step, eliminating the need for any hyper-parameter tuning. Using the framework we propose a cyclic majorization-minimization based technique and apply it to synthetic data to evaluate its performance in terms of normalized root mean square error (NRMSE) and Kullback Leibler (KL) divergence.

Multistatic localization techniques employ noisy range measurements collected via multiple transmitters and receivers to localize a target. However, in many realistic scenarios the data are corrupted by outliers which may be due to the failure of or malicious attack on one or more sensors. The presence of outliers leads to performance degradation in terms of target localization accuracy. In this letter, we address the problem of multistatic target localization when the measurements contain outliers. We employ a multi-hypothesis testing method based on the false discovery rate (FDR) to detect the outliers. More specifically, we consider a penalized maximum likelihood problem for joint estimation of the number and positions of the outliers as well as the target position, and the noise variance. To solve this problem, an iterative algorithm employing the majorization-minimization technique that minimizes the objective in a monotonic manner is developed. Through numerical simulations, we compare the proposed algorithm with other robust state-of-the-art algorithms and show that the proposed algorithm has superior performance.

This paper focuses on the problem of three-dimensional (3D) frequency retrieval from a set of time samples. The assumption made in this study is that the signal being analyzed consists of K continuous-valued 3D complex sinusoidal signals. The main objective is to identify all frequency components when only one snapshot of regularly-spaced time samples is available. The vectorized version of atomic norm minimization (ANM) method could accurately solve the problem. Nonetheless, this technique demands significant computational resources. In this paper, we propose a computationally-efficient method based on atomic norm minimization known as three-dimensional decoupled ANM (3D-DANM). This approach transforms the estimation problem into a semi-definite programming (SDP) problem. By performing Vandermonde decomposition on the Toeplitz matrices resulted from the SDP problem, the frequencies of all sources will be estimated. The extracted parameters are then paired using a specific approach. Simulation results demonstrate that the proposed method closely approaches the Cramér-Rao lower bound (CRLB) for sufficient signal-to-noise ratio (SNR) values. Additionally, it offers significant improvements in computational cost compared to other existing approaches.

We present a block minorization-maximization (MM) algorithm to solve the row-sparse principal component analysis (RSPCA) problem. The RSPCA problem consists of orthogonality and row-sparsity constraints. We model the decision variable as a product of a selection matrix and the matrix of principal components. This problem is solved by updating the two blocks in a cyclic manner. As the problem with respect to the selection matrix does not admit a closed-form solution, we propose to utilize the MM technique to solve this subproblem. Numerical simulations are provided to show the efficacy of the proposed algorithm.

This work considers Maximum Likelihood Estimation (MLE) of a Toeplitz structured covariance matrix. In this regard, an equivalent reformulation of the MLE problem is introduced, and two iterative algorithms are proposed for the optimization of the equivalent statistical learning framework. Both strategies are based on the Majorization Minimization (MM) paradigm and hence enjoy nice properties such as monotonicity and ensured convergence to a stationary point of the equivalent MLE problem. The proposed framework is also extended to deal with MLE of other practically relevant covariance structures, namely, the banded Toeplitz, block Toeplitz, and Toeplitz-block-Toeplitz. Through numerical simulations, it is shown that the new methods provide excellent performance levels in terms of both mean square estimation error (which is very close to the benchmark Cramér-Rao Bound (CRB)) and signal-to-interference-plus-noise ratio, especially in comparison with state-of-the art strategies. Moreover, the estimation task is accomplished with a remarkable reduction in computational complexity compared with a standard approach relying on a Semidefinite Programming (SDP) solver.

In this letter we revisit the problem of smoothed nonparametric spectral estimation via cepstrum thresholding. We formulate the problem of cepstrum thresholding as a multiple hypothesis testing problem and use the false discovery rate (FDR) and familywise error rate (FER) procedures to threshold the cepstral coefficients. We compare the FDR and FER approaches with a previously proposed individual hypothesis testing approach and show that the cepstrum thresholding based on FDR and FER can yield spectral estimates with lower mean square error (MSE).

In this letter, we propose an algorithm for learning a sparse weighted graph by estimating its adjacency matrix under the assumption that the observed signals vary smoothly over the nodes of the graph. The proposed algorithm is based on the principle of majorization-minimization (MM), wherein we first obtain a tight surrogate function for the graph learning objective and then solve the resultant surrogate problem which has a simple closed form solution. The proposed algorithm does not require tuning of any hyperparameter and it has the desirable feature of eliminating the inactive variables in the course of the iterations - which can be used to speed up the algorithm. The numerical simulations conducted using both synthetic and real world (brain-network) data show that the proposed algorithm converges faster, in terms of the average number of iterations, than several existing methods in the literature.

In this paper, we propose an iterative algorithm named PGPAL for the phase-retrieval problem under the scenario when the measurements follow Poisson plus Gaussian (PG) distribution, which is a more realistic model in applications like astronomy, microscopy, medical imaging, and remote sensing. The proposed algorithm is based on majorization-minimization (MM) framework, in which a simple surrogate function that tightly upperbounds the challenging underlying maximum-likelihood (ML) objective is constructed and minimized iteratively. The proposed algorithm monotonically decreases the ML objective and the algorithm is guaranteed to converge to a stationary point of the ML objective. Numerical simulations are performed for the one-dimensional phase-retrieval case, and the performance of the proposed method is compared with the recently proposed Poisson phase-retrieval algorithm, which assumes the measurement model to be only Poisson distributed. The performance of the proposed MM-based algorithm in terms of the normalized root mean square error (NRMSE) is found to be much better in comparison to the method for Poisson phase-retrieval.

In this letter we propose a new method to estimate the covariance matrix under the constraint that its off-diagonal elements are non-negative, which has applications to portfolio selection in finance. We incorporate the non-negativity constraint in the maximum likelihood (ML) estimation problem and propose an algorithm based on the block coordinate descent method to solve for the ML estimate. To study the effectiveness of the proposed algorithm, we perform numerical simulations on both synthetic and real-world financial data, and show that our proposed method has better performance than that of a state-of-the-art method.

In this letter, we address the problem of multistatic target localization using time-sum-of-arrival (TSOA) measurements in the presence of errors due to non-line-of-sight (NLOS) propagation and outliers. To handle the effect of bias errors induced due to NLOS/outliers, an additional balancing parameter is incorporated in the data model. Then, a non-linear weighted least squares (NLWLS) based design objective is formulated for the joint estimation of source position and the balancing parameter. We then propose a simple iterative algorithm, which enjoys closed-form updates for the source position and the balancing parameter, based on the majorization-minimization (MM) technique to arrive at a minimizer of the NLWLS objective. Various numerical simulations are presented in the letter to demonstrate the accuracy of the proposed algorithm when compared to other existing algorithms in the literature.

In this letter we address the problem of multistatic target localization using time-delay measurements corrupted by noise with unknown non-uniform variances. More concretely, we consider the problem of joint maximum likelihood (ML) estimation of the target position and the noise variances, for which we propose a majorization-minimization based algorithm. The proposed approach is compared with state-of-the-art algorithms, and the simulation results show the excellent accuracy of our algorithm.

We consider the problem of localizing the source using range, and range-difference measurements. Both the problems are non-convex, and non-smooth, and are challenging to solve. In this article, we develop an iterative algorithm - Source Localization Via an Iterative technique (SOLVIT) to localize the source using all the distinct range-difference measurements, i.e., without choosing a reference sensor. SOLVIT is based on the Majorization Minimization approach - in which a novel upper bound is formulated, and minimized to get a closed-form solution at every iteration. We also solve the source localization problem based on range measurements, and rederive the Standard Fixed Point algorithm using the Majorization Minimization approach. By doing so, we show a less intricate way to prove the convergence of the Standard Fixed Point algorithm. Numerical simulations, and experiments in an anechoic chamber confirm that SOLVIT performs better than existing reference-based, and reference-free methods in terms of source positioning accuracy.

In this paper, we study the problem of downlink (DL) sum rate maximization in codebook based multiuser (MU) multiple input multiple output (MIMO) systems. The user equipments (UEs) estimate the DL channels using pilot symbols sent by the access point (AP) and feedback the estimates to the AP over a control channel. We present a closed form expression for the achievable sum rate of the MU-MIMO broadcast system with codebook constrained precoding based on the estimated channels, where multiple data streams are simultaneously transmitted to all users. Next, we present novel, computationally efficient, minorization-maximization (MM) based algorithms to determine the selection of beamforming vectors and power allocation to each beam that maximizes the achievable sum rate. Our solution involves multiple uses of MM in a nested fashion. Based on this approach, we propose and contrast two algorithms, which we call the square-root-MM (SMM) and inverse-MM (IMM) algorithms. The algorithms are iterative and converge to a locally optimal beamforming vector selection and power allocation solution from any initialization. We evaluate the performance and complexity of the algorithms for various values of the system parameters, compare them with existing solutions, and provide further insights into how they can be used in system design.

The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulated this problem by adding a penalty term into the objective function that encourages a sparse solution. Nevertheless, the resulting methods achieve sparsity at a sacrifice of the orthogonality property. In this paper, we develop a new method to estimate dominant sparse eigenvectors without trading off their orthogonality. The problem is highly non-convex and too hard to handle. We apply the minorization-maximization (MM) framework where we iteratively maximize a tight lower bound (surrogate function) of the objective function over the Stiefel manifold. The inner maximization problem turns out to be the rectangular Procrustes problem, which has a closed-form solution. Numerical experiments show that the propose method matches or outperforms existing algorithms in terms of recovery probability and explained variance.

Sequences with low aperiodic autocorrelation sidelobes are well-known to have extensive applications in active sensing and communication systems. In this paper, we consider the sequence design problem of minimizing the ℓp-norm of the autocorrelation side-lobes, which can then be used to minimize the peak sidelobe level (PSL) criterion. An algorithm based on the general majorization-minimization method is developed to tackle the problem. The proposed algorithm can be implemented by means of fast Fourier transform (FFT) operations and thus is computationally efficient in practice. Numerical experiments show that the proposed algorithm can produce very long sequences with impulse-like autocorrelation and with much smaller PSL compared with some well-known analytical sequences.

Unimodular sequences have been widely used in communications and radars, for which some numerical algorithms have been proposed recently to obtain good autocorrelation properties [1,2]. Design of such "good" sequences, however, does not take into account any prior information of the channel to be estimated. Although shaping the autocorrelation of a training sequence may imply a good performance, it may be advantageous to directly optimize the performance measure of interest. In this paper, we consider the problem of optimal constant-modulus training sequence design for MMSE estimation of the channel impulse response and conditional mutual information maximization. Efficient iterative algorithms based on the majorization-minorization framework are proposed for each formulation. Numerical examples show that our proposed training sequences achieve better performances than that of low sidelobes or random phases.

In this letter, we develop a low-complexity algorithm named maximum-likelihood estimation of low-rank Toeplitz covariance matrix (MELT) to solve the maximum-likelihood estimation of a low-rank Toeplitz covariance matrix. Our derivation of MELT is based on the technique of majorization-minimization (MM), in which we design and optimize a novel tight upper-bound function. MELT is an iterative algorithm, and its each iterative step is a closed-form update, which can be implemented efficiently by fast Fourier transforms. As MELT is based on MM, it enjoys nice properties such as monotonicity and guaranteed convergence to a stationary point. Finally, we numerically show that the performance of MELT is much better than some of the algorithms currently available in the literature.

Unimodular sequences with low autocorrelations are desired in many applications, especially in the area of radar and code-division multiple access (CDMA). In this paper, we propose a new algorithm to design unimodular sequences with low integrated sidelobe level (ISL), which is a widely used measure of the goodness of a sequence's correlation property. The algorithm falls into the general framework of majorization-minimization (MM) algorithms and thus shares the monotonic property of such algorithms. In addition, the algorithm can be implemented via fast Fourier transform (FFT) operations and thus is computationally efficient. Numerical experiments show that the proposed algorithm outperforms the state-of-the-art algorithm in terms of both the quality of designed sequences and the computational complexity.

In this paper, we consider the robust covariance estimation problem in the non-Gaussian set-up. In particular, Tyler's M-estimator is adopted for samples drawn from a heavy-tailed elliptical distribution. For some applications, the covariance matrix naturally possesses certain structure. Therefore, incorporating the prior structure information in the estimation procedure is beneficial to improving estimation accuracy. The problem is formulated as a constrained minimization of the Tyler's cost function, where the structure is characterized by the constraint set. A numerical algorithm based on majorization-minimization is derived for general structures that can be characterized as a convex set, where a sequence of convex programming is solved. For the set of matrices that can be decomposed as the sum of rank one positive semidefinite matrices, which has a wide range of applications, the algorithm is modified with much lower complexity. Simulation results demonstrate that the proposed structure-constrained Tyler's estimator achieves smaller estimation error than the unconstrained case.

Model selection is an important task in many signal processing applications. In this letter, we present a generalized likelihood ratio (GLR)-based derivation of the recently proposed EEF rule in an attempt to cast EEF in the main stream of model order selection approaches and provide further insights into its theoretical foundations. We also show that EEF can be expected to behave asymptotically (in the number of data samples) similarly to the Bayesian information criterion (BIC). To evaluate the finite sample performance we consider two numerical examples, including the selection of the number of components in a Gaussian mixture model (GMM), by means of which we show that EEF behaves similarly to BIC.

In this paper we introduce a new technique for estimating the parameters of the Keplerian model commonly used in radial velocity data analysis for extrasolar planet detection. The un known parameters in the Keplerian model, namely eccentricity e, orbital frequency f, periastron passage time T, longitude of periastron ω, and radial velocity amplitude K are estimated by a new approach named SPICE (a semi-parametric iterative covariance-based estimation technique). SPICE enjoys global convergence, does not require selection of any hyperparameters, and is computationally efficient (indeed computing the SPICE estimates boils down to solving a numerically efficient linear program (LP)). The parameter estimates obtained from SPICE are then refined by means of a relaxation-based maximum likelihood algorithm (RELAX) and the significance of the resultant estimates is determined by a generalized likelihood ratio test (GLRT). A real-life radial velocity data set of the star HD 9446 is analyzed and the results obtained are compared with those reported in the literature.

In this paper we present a new sparse iterative covariance-based estimation approach, called SPICE, to the direction of arrival estimation problem. SPICE is obtained by the minimization of a statistically well motivated covariance matrix fitting criterion and can be used in both single and multiple-snapshot cases. Some of the unique features enjoyed by SPICE are: it takes account of the noise in the data in a natural manner, it does not require selection of any hyper-parameters, and it has global convergence properties.

In this paper, we show that Sudoku puzzles can be formulated and solved as a sparse linear system of equations. We begin by showing that the Sudoku ruleset can be expressed as an underdetermined linear system: Ax = b, where A is of size m times n and n > m. We then prove that the Sudoku solution is the sparsest solution of Ax = b, which can be obtained by lo norm minimization, i.e. min ||x:||0s.t. Ax = b. Instead of this minimization SB problem, inspired by the sparse representation literature, we solve the much simpler linear programming problem of minimizing the l1norm of x, i.e. min ||x||1s.t. Ax = b, and show numerically that this approach solves representative Sudoku puzzles.

In this note, we show that the iterative frequency estimation technique proposed in is nothing but an approximate Gauss-Newton (GN) algorithm for minimizing the standard nonlinear least squares fitting criterion. Starting from the complex sinusoidal data model used in , we derive the GN algorithm for frequency estimation and show that by approximating some steps in the GN algorithm we arrive at the estimator proposed in . We then numerically compare the performances of GN and approximate GN algorithms with that of zero padded FFT.

Elfving theorem is a fundamental result in the area of optimal experiment design, and yet its available proofs require a number of somewhat indirect geometrical arguments that might detract a potential user from its full understanding and exploitation. In this letter, we provide a direct algebraic proof of this theorem. Furthermore, we make some connections with the ℓ1- norm minimization approach commonly used for sparse estimation, which suggest importation of algorithms and results from the latter area into that of optimal experiment design.