Covariance matrix estimation in high dimensions is a fundamental problem in machine learning and signal processing. A common structural assumption used to mitigate the challenges posed by high dimensionality is sparsity, which posits that most variable pairs exhibit negligible correlations. In this paper, we revisit the classical problem of positive definite sparse covariance estimation (PDSCE) introduced by Rothman (2012). Unlike many earlier approaches, this formulation incorporates a logarithmic barrier, which guarantees that the resulting covariance estimator is positive definite and thereby ensures the well-posedness of the estimation problem. However, the inclusion of the logarithmic barrier also leads to nontrivial optimization difficulties. To overcome these difficulties, we propose a dual proximal gradient method (DPGM) for solving the PDSCE problem. In contrast to existing primal-space approaches, DPGM operates directly in the dual space. This dual perspective provides several key advantages. First, DPGM significantly reduces computational costs, because positive definiteness is preserved automatically and no iterative subproblem solvers are required. Second, compared with primal optimization algorithms, DPGM offers stronger theoretical guarantees, including principled step size selection and improved iteration complexity. Extensive numerical experiments demonstrate that DPGM consistently outperforms existing methods, which confirms its effectiveness and scalability for high-dimensional sparse covariance estimation.

Covariance matrix estimation is a fundamental problem in multivariate data analysis, which becomes particularly challenging in high-dimensional settings due to the curse of dimensionality. To enhance estimation accuracy, structural regularization is often imposed on the precision matrix (the inverse covariance matrix) for covariance selection. In this paper, we study covariance selection in a distributed setting, where data is spread across a network of agents. We formulate the problem as a Gaussian maximum likelihood estimation problem with structural penalties and propose a novel algorithmic framework called NetGGM. Unlike existing methods that rely on a central coordinator, NetGGM operates in a fully decentralized manner with low computational complexity. We provide theoretical guarantees showing that NetGGM converges linearly to the global optimum while ensuring consensus among agents. Numerical experiments validate its convergence properties and demonstrate that it outperforms state-of-the-art methods in precision matrix estimation.

The generalized hyperbolic (GH) distribution is a highly flexible probability distribution that finds applications in various fields, yet estimating its parameters is quite challenging. This paper focuses on the maximum likelihood (ML) estimation of the GH distribution. In the literature, several expectation-maximization (EM) type algorithms have been proposed, which iteratively optimize a surrogate objective called the Q-function—a tractable lower bound of the likelihood function. While these generic EM-type algorithms, derived based on Jensen’s inequality, provide a systematic framework, they often restrict flexibility in algorithm design. In this study, we adopt the block minorization-maximization (BMM) framework, a more general iterative surrogate maximization approach that subsumes EM-type algorithms as special cases, to address the ML estimation problem. We propose efficient, problem-specific algorithms that utilize novel surrogate functions, which provide a provably tighter surrogate function on the likelihood than the Q-function while still allowing for closed-form updates. As a result, the proposed algorithm achieves faster and guaranteed convergence. Numerical experiments on synthetic data confirm the superior convergence speed of our algorithms compared to existing ones.

In this paper, we study the sparse principal component analysis (PCA) problem in high-dimensional settings. We propose a novel row-sparse principal subspace estimator, which estimates the subspace spanned by multiple eigenvectors based on the variance maximization problem with a nonconvex sparsity regularizer. To tackle the nonconvex estimation problem, we introduce a minorization-maximization (MM) algorithm to decompose it into a sequence of convex subproblems. Each subproblem is solved based on the alternating direction method of multipliers. Theoretically, we provide a comprehensive analysis of both the computational and statistical properties of the iterates from the MM algorithm. We demonstrate that the proposed sparse PCA estimator can achieve the same statistical rate as the oracle estimator. Simulation results corroborate the theoretical findings and highlight the superiority of the proposed sparse PCA estimator.

This paper addresses the problem of covariance matrix estimation in scenarios where the underlying variables can be divided into groups, with variables within each group being highly correlated. Consequently, the covariance matrix displays both sparse and approximately low-rank characteristics due to these highly correlated groups. By appropriately rearranging the variables, the covariance matrix can be transformed into an approximately block diagonal form. In this work, we investigate the estimation of covariance matrices under this structure in high dimensions. We propose a least squares-based covariance estimation method that incorporates a trace norm along with a nonconvex sparsity regularizer to promote both low-rankness and sparsity. Additionally, we introduce a spectral constraint to ensure the positive semi-definiteness of the covariance matrix, even in cases of finite samples, while permitting the integration of prior spectral information. To solve this nonconvex statistical estimation problem, we develop an algorithm based on the majorization-minimization framework, which iteratively solves a convex subproblem. We provide theoretical guarantees that demonstrate the proposed algorithm converges to an estimator achieving the oracle statistical rate under mild technical conditions. Numerical experiments corroborate these theoretical findings.

The covariance matrix is a fundamental secondorder statistic in signal and information processing that quantifies the linear relationships among multiple variables. When data evolves rapidly, or the acquisition devices have limited processing power and storage-particularly given the large dimensionality of modern datasets-covariance estimation becomes challenging. To address these challenges, it is desirable to estimate the covariance matrix from a single pass over the data with compressive measurements. In this paper, we study covariance matrix estimation in high dimensions based on quadratic measurements, assuming that the covariance matrix exhibits a sparse structure. We formulate the problem as a least-squares estimation with nonconvex sparsity-inducing penalties. To efficiently compute this estimator, we develop a multi-stage convex relaxation algorithm based on the majorization-minimization algorithmic framework. We comprehensively characterize the computational and statistical properties of the iterates from the algorithm and show that the estimator from the proposed method achieves the oracle statistical rate of convergence after sufficient iterations. Numerical simulations support the theoretical findings and validate the performance of the proposed estimator.

This paper studies the problem of recovering a low-rank matrix from noisy bilinear measurements, which arises in a range of real-world applications. We propose a novel estimator that minimizes a least-squares loss regularized by a nonconvex penalty to promote low-rank structure. To solve the resulting nonconvex problem, we develop an efficient proximal gradient descent algorithm. We show that, under mild conditions, the proposed estimator consistently recovers the underlying matrix and achieves the statistically optimal convergence rate. Numerical experiments on both synthetic and real-world datasets validate the theoretical guarantees and demonstrate the practical effectiveness of the proposed method.

The problem of joint blind deconvolution and demixing for sparse signals is prevalent in many signal processing areas. The goal of this problem is to recover both the sparse signals and the filters from a noisy mixture of bilinear measurements. Due to the bilinear factorization structure, the common solving approach is based on the matrix-lifting semidefinite programming. In this paper, we consider a nonconvex problem formulation for this problem based on matrix factorization. We propose a nonconvex optimization algorithm based on the block majorization-minimization (BMM) framework. In each iteration of BMM, signals and filters are updated with analytical solutions in an alternating way. In comparison to state-of-the-art algorithms, BMM has much lower per-iteration computational complexity and hence is more scalable to large-size problems. Numerical results show that BMM is able to recover the sparse signals and filters with higher precision as well as faster convergence compared to existing methods.

We present an accelerated method for the asymmetric low-rank matrix sensing problem in the over-parameterized setup, named preconditioned gradient descent. We analyze the local convergence rate of the proposed algorithm starting from spectral initialization. Our algorithm is shown to have linear convergence rate independent of condition number even when ill-conditioning and over-parameterization both exist in the asymmetric matrix sensing problem. Numerical results verify the theoretical findings and demonstrate the performance of the proposed algorithm.

In this paper, we study the problem of large covariance matrix estimation based on the factor model assumption, in which case the covariance matrix is represented by a combination of a low-rank matrix and a sparse matrix. We formulate the estimation problem as a nonconvex problem, and an iterative optimization algorithm is proposed to obtain the estimator. The algorithm starts at the widely used optimization-free estimator called principal orthogonal complement thresholding (POET), and then a refined estimator is obtained by iterative optimization. We name the obtained nonconvex estimator POET with refining iteratively (POETRY). Theoretically, we prove that POETRY achieves a superior statistical rate compared to POET, matching the minimax rate of convergence for factor-based covariance estimation. Additionally, our algorithm exhibits significantly lower per-iteration computational complexity compared to existing convex relaxation-based methods. Numerical experiments validate the superiority of our algorithm over the state-of-the-art ones and corroborate our theory.

We consider the problem of two-way sparse reduced-rank regression (TSRRR). The purpose of TSRRR is to estimate the coefficient matrix in the multiple response linear regression model where the coefficient matrix is simultaneously low-rank and two-way sparse (i.e., row and column sparse). In this work, we formulate TSRRR as a nonconvex optimization problem and propose an efficient and scalable algorithm dubbed as ScaledGDT (Scaled Gradient Descent with hard Thresholding). To demonstrate the efficiency and scalability of our proposed algorithm, we prove the linear convergence rate which is independent of the condition number of the coefficient matrix obtained by the iterates of ScaledGDT, to the region within statistical error up to the optimal solution. Also, the statistical error rate obtained by ScaledGDT is verified to be near optimal compared to minimax rate, which confirms its satisfactory estimation accuracy. Sim-ulations validate the competitive performance of our proposed algorithm compared with existing methods.

Large precision matrix estimation is a crucial problem in high-dimensional multivariate data analytics. When the data exhibit heavy-tailedness or outliers, existing estimators based on Gaussian distribution assumption become inefficient. In this paper, we introduce tLasso, a novel adaptive model for robust precision matrix estimation based on an$\ell_{1}$-penalized t-distribution log-likelihood function. We assume the degree of freedom parameter of the t-distribution to be unknown a priori, adapting to the arbitrary heavy-tailedness of data distributions. Then we propose a regularized MCECM algorithm to solve the non-convex estimation problem. Theoretically, we prove the non-asymptotic bound on the estimation error, which establishes the computational and statistical guarantees of the algorithm. Numerical simulations validate the effectiveness of the proposed estimator.

Fractional programming (FP) arises in various communications and signal processing problems because several key quantities in the field are fractionally structured, e.g., the Cramér-Rao bound, the Fisher information, and the signal-to-interference-plus-noise ratio (SINR). A recently proposed method called the quadratic transform has been applied to the FP problems extensively. The main contributions of the present paper are two-fold. First, we investigate how fast the quadratic transform converges. To the best of our knowledge, this is the first work that analyzes the convergence rate for the quadratic transform as well as its special case the weighted minimum mean square error (WMMSE) algorithm. Second, we accelerate the existing quadratic transform via a novel use of Nesterov's extrapolation scheme [2]. Specifically, by generalizing the minorization-maximization (MM) approach in [3], we establish a nontrivial connection between the quadratic transform and the gradient projection, thereby further incorporating the gradient extrapolation into the quadratic transform to make it converge more rapidly. Moreover, the paper showcases the practical use of the accelerated quadratic transform with two frontier wireless applications: integrated sensing and communication (ISAC) and massive multiple-input multiple-output (MIMO).

In this paper, we study the classic problem of robust M-estimation of a location parameter. This problem involves minimizing a finite sum of non-convex loss functions. We investigate the geometric structure of the empirical non-convex objective. Under certain assumptions, we prove that the optimization landscape can be characterized by two favorable regions: a strong convex region within a ball centered at the minimum and a one-point strong convex region outside a ball centered at the minimum. Utilizing these results, we establish conditions under which the typically non-convex estimation problem possesses a unique global minimum that is close to the ground truth. By exploiting the favorable landscape properties, numerical methods such as gradient descent can achieve global convergence to the unique optimum from any starting point. Our theoretical conclusions are supported by numerical experiments.

Graph convolutional neural networks (GCNNs) have emerged as powerful tools for processing signals or data supported by graphs. However, their effectiveness is compromised when perturbations exist in the graph structures. While previous research has studied the stability of GCNNs against such perturbations by analyzing the behavior of the graph convolutional filters, we found a significant discrepancy exists between the theoretical stability bounds and simulation outcomes. In this paper, we propose a novel approach to characterize the stability of GCNNs more accurately. Unlike existing methods that treat graph convolutional filters in each layer of a GCNN as separate SISO systems, our approach views them as a compact MIMO system. This perspective yields an improved stability characterization that aligns more closely with empirical observations and provides insights into designing GCNNs that are more resilient to graph perturbations. Numerical experiments on synthetic data confirm our theoretical findings and experiments on a movie recommendation problem demonstrate how it helps train stable GCNNs.

Fractional programming (FP) arises in various communications and signal processing problems because several key quantities in these fields are fractionally structured, e.g., the Cramér-Rao bound, the Fisher information, and the signal-to-interference-plus-noise ratio (SINR). A recently proposed method called the quadratic transform has been applied to the FP problems extensively. The main contributions of the present paper are two-fold. First, we investigate how fast the quadratic transform converges. To the best of our knowledge, this is the first work that analyzes the convergence rate for the quadratic transform as well as its special case the weighted minimum mean square error (WMMSE) algorithm. Second, we accelerate the existing quadratic transform via a novel use of Nesterov’s extrapolation scheme. Specifically, by generalizing the minorization-maximization (MM) approach, we establish a subtle connection between the quadratic transform and the gradient projection, thereby further incorporating the gradient extrapolation into the quadratic transform to make it converge more rapidly. Moreover, the paper showcases the practical use of the accelerated quadratic transform with two frontier wireless applications: integrated sensing and communications (ISAC) and massive multiple-input multiple-output (MIMO).

The ℓ1penalized covariance estimator has been widely used for estimating large sparse covariance matrices. It was recognized that ℓ1penalty introduces a non-negligible estimation bias, while a proper utilization of non-convex penalty may lead to an estimator with a refined statistical rate of convergence. In this paper, to eliminate the estimation bias we propose to estimate large sparse covariance matrices using the non-convex penalty. It is a challenging task to analyze the theoretical properties of the resulting covariance estimator because popular iterative algorithms for convex optimization no longer have global convergence guarantees for non-convex optimization. To tackle this issue, an efficient algorithm based on the majorization-minimization (MM) is developed by solving a sequence of convex relaxation subproblems. We prove that the proposed estimator computed exactly by the MM-based algorithm achieves the oracle statistical rate under weak assumptions. Our theoretical findings are corroborated through extensive numerical experiments.

Weighted minimum mean squared error (WMMSE) and fractional programming (FP) constitute two common approaches to the weighted sum-rate maximization in communication system design. One subtle issue with WMMSE and FP lies in the tuning of a Lagrange multiplier for the power constraint when it comes to the multi-antenna transmission. To obtain the optimal Lagrange multiplier, we must repeatedly inverse an M × M matrix, where M is the number of transmit antennas, which incurs considerable complexity. To address the above issue, this work explores the connection of WMMSE and FP to minorization-maximization (MM), thereby modifying the two methods to get rid of the Lagrange multiplier. The proposed algorithm enables a parameter-free iterative optimization of the beamforming vectors with the power constraint enforced automatically. Numerical results demonstrate the faster convergence of the proposed beamforming method as compared to the conventional WMMSE and FP methods.

Normalized difference vegetation index (NDVI) is widely utilized to examine vegetation coverage and estimate crop yield. To obtain a high-resolution (HR) NDVI, fusion techniques, which first generates a HR multispectral (MS) image by fusing a low-resolution (LR) MS image and a HR panchromatic image, and then calculates the HR NDVI based on the fused HR MS image, are utilized in previous studies. A HR vegetation index calculated on the basis of HR panchromatic image could provide HR spatial resolution, and this vegetation index has a spatial structure that is similar to that of NDVI. Therefore, this similarity is investigated to construct a novel method called FusionNDVI to improve the fusion performance in this study. The fusion problem is formulated to minimize a least square fitting error term and a nonlocal gradient sparsity regularization term. The fitting term is used to limit the difference between the fused HR NDVI and the LR NDVI, whereas the regularizer enforces a similar nonlocal spatial structure in the fused NDVI and the HR vegetation index. An efficient solving algorithm based on the augmented Lagrangian method of multipliers is derived. The superiority of the proposed FusionNDVI method over the state-of-the-art ones is verified via simulations.

In this paper, a gradient-based method for bound constrained non-convex problems is proposed. By leveraging both projected gradient descent and perturbed gradient descent, the proposed algorithm, named perturbed projected gradient descent (PP-GD), converges to some approximate second-order stationary (SS2) points (which satisfy certain approximate second-order necessary conditions) with provable convergence rate guarantees. The proposed algorithm is suitable for a large-scale problem since it only uses the gradient information of the objective function. It also seamlessly incorporates variable constraints such as nonnegativity, which is commonly seen in many practical machine learning problems. We provide a concrete theoretical analysis showing that PP-GD is able to obtain approximate second-order solutions by extracting the negative curvature of the objective function around the strict saddle points. Numerical results demonstrate that PP-GD indeed converges faster compared to other first-order methods in the presence of strict saddle points.

Minimax multiple-input multiple-output (MIMO) transmit beampattern matching is a fundamental and important problem in many MIMO systems. The problem is formulated to minimize the maximum beampattern matching error as well as suppress the cross-correlation beampatterns while taking different practical waveform constraints into consideration. Due to the high nonconvexity of the problem, the traditional way for problem solving is a two-stage approach, where a waveform covariance matrix is firstly designed and then the waveforms are synthesized from the covariance matrix under a specific constraint. This approach is usually very time consuming and only results in suboptimal solutions. In this paper, a novel and unified one-stage approach is proposed to solve the minimax beampattern matching problem which is capable of considering multiple waveform constraints. Superior performance of the proposed approach over the classical approach is verified through numerical simulations.

In this paper, the multiple-input multiple-output (MIMO) transmit beampattern matching problem is considered. The problem is formulated to approximate a desired transmit beampattern (i.e., an energy distribution in space and frequency) and to minimize the cross-correlation of signals reflected back to the array by considering different practical waveform constraints at the same time. Due to the nonconvexity of the objective function and the waveform constraints, the optimization problem is highly nonconvex. An efficient one-step method is proposed to solve this problem based on the majorization-minimization (MM) method. The performance of the proposed algorithms compared to the state-of-art algorithms is shown through numerical simulations.