We address the Normalized Signal to Noise Ratio (NSNR) metric defined in the seminal paper by Reed, Mallett, and Brennan on adaptive detection. NSNR is the ratio between the SNR of a linear detector which uses an estimated noise covariance and the SNR of a clairvoyant detector based on the exact unknown covariance. It is not obvious how to evaluate NSNR since it is a function of the target vector. To close this gap, we consider the NSNR associated with the worst target. Using the Kantorovich Inequality, we provide a closed-form solution for the worst-case NSNR. Then, we prove that the classical Gaussian Kullback Leibler (KL) divergence bounds it. Motivated by these results, we derive a simple variant of a classic norm based estimator by incorporating KL in a leave-one-out cross-validation (LOOCV) framework. Numerical experiments with different true covariances and various estimates suggest that the KL metric is more correlated with the NSNR metric than competing norm-based metrics and simply changing the metric in the LOOCV estimator improves KL and NSNR performance.

We consider the problem of learning uncertainty regions for parameter estimation problems. The regions are ellipsoids that minimize the average volumes subject to a prescribed coverage probability. As expected, under the assumption of jointly Gaussian data, we prove that the optimal ellipsoid is centered around the conditional mean and shaped as the conditional covariance matrix. In more practical cases, we propose a differentiable optimization approach for approximately computing the optimal ellipsoids using a neural network with proper calibration. Compared to existing methods, our network requires less storage and less computations in inference time, leading to accurate yet smaller ellipsoids. We demonstrate these advantages on four real-world localization datasets.

In this paper we consider the problem of linear unmixing hidden random variables defined over the simplex with additive Gaussian noise, also known as probabilistic simplex component analysis (PRISM). Previous solutions to tackle this challenging problem were based on geometrical approaches or computationally intensive variational methods. In contrast, we propose a conventional expectation maximization (EM) algorithm which embeds importance sampling. For this purpose, the proposal distribution is chosen as a simple surrogate distribution of the target posterior that is guaranteed to lie in the simplex. It is based on fitting the Dirichlet parameters to the linear minimum mean squared error (LMMSE) approximation, which is accurate at high signal-to-noise ratio. Numerical experiments in different settings demonstrate the advantages of this adaptive surrogate over state-of-the-art methods.

Heavy tailed and contaminated data are common in various applications of machine learning. A standard technique to handle regression tasks that involve such data, is to use robust losses, e.g., the popular Huber’s loss. In structured problems, however, where there are multiple labels and structural constraints on the labels are imposed (or learned), robust optimization is challenging, and more often than not the loss used is simply the negative log-likelihood of a Gaussian Markov random field. Heavy tailed and contaminated data are common in various applications of machine learning. A standard technique to handle regression tasks that involve such data, is to use robust losses, e.g., the popular Huber’s loss. In structured problems, however, where there are multiple labels and structural constraints on the labels are imposed (or learned), robust optimization is challenging, and more often than not the loss used is simply the negative log-likelihood of a Gaussian Markov random field. In this work, we analyze robust alternatives. Theoretical understanding of such problems is quite limited, with guarantees on optimization given only for special cases and non-structured settings. The core of the difficulty is the non-convexity of the objective function, implying that standard optimization algorithms may converge to sub-optimal critical points. Our analysis focuses on loss functions that arise from elliptical distributions, which appealingly include most loss functions proposed in the literature as special cases. We show that, even though these problems are non-convex, they can be optimized efficiently. Concretely, we prove that at the limit of infinite training data, due to algebraic properties of the problem, all stationary points are globally optimal. Finally, we demonstrate the empirical appeal of using these losses for regression on synthetic and real-life data.

In this paper, we consider the parameter estimation of K-Gaussians, given convex combinations of their realizations. In the remote sensing literature, this setting is known as the normal compositional model (NCM) and has shown promising gains in modeling hyperspectral images. Current NCM parameter estimation techniques are based on Bayesian methodology and are computationally slow and sensitive to their prior assumptions. Here, we introduce a deterministic variant of the NCM, named DNCM, which assumes that the unknown mixing coefficients are nonrandom. This leads to a standard Gaussian model with a simple estimation procedure, which we denote by K-Gaussians. Its iterations are provided in closed form and do not require any sampling schemes or simplifying structural assumptions. We illustrate the performance advantages of K-Gaussians using synthetic and real images, in terms of accuracy and computational costs in comparison to state of the art. We also demonstrate the use of our algorithm in hyperspectral target detection on a real image with known targets.

We consider the problem of hypothesis testing for detection of a signal in Gaussian noise. We assume that the vector of measurements is unobserved, and that our observations consist of phaseless inner products with a set of known measurement vectors. This is typical of the phase retrieval problem, where the goal is to recover the vector of measurements. We provide a simple estimator for the test statistic that does not necessitate a phaseless recovery method to reconstruct the measurements. Our analysis shows that for random measurement vectors, we can reconstruct the test statistic for any signal from a sufficient number of observations, quadratic in the signal length, using a simple least-squares approach. The primary advantage of this method its simplicity and computational efficiency, which comes at the expense of requiring many more measurements. We show that for Fourier measurements vectors, our approach works only when the signal is also a Fourier vector.

In this paper we address target detection in correlated non-Gaussian noise. We introduce a powerful class of multivariate complex valued distribution that allows us to specify flexible non-Gaussian marginals, as well as correlation between the variables, while preserving circular symmetry. For noise belonging to this class, we study the fundamental problem of signal detection under different settings, and develop the needed (generalized) likelihood ratio tests. We also consider the problem of estimation of the noise parameters, and derive the maximum likelihood formulations. We compare the performance of the proposed methods using numerical simulations on synthetic data, and demonstrate the importance of using both correlations and non-Gaussiantiy.

In this paper we study the hybrid precoding design problem for a frequency selective massive MIMO channel, e.g., the millimeter wave (mmWave) massive MIMO channel. In contrast to a traditional MIMO system, a hybrid analog-digital MIMO scheme is preferred for massive MIMO systems due to the high cost and power consumption of the radio frequency (RF) chains. The RF analog precoding is implemented using only phase shift networks, which impose constant modulus constraints on the RF precoding and decoding matrices. Moreover, there is just one common equivalent RF beamforming matrix for all subcarriers. The resulting sum rate maximization problem is non-convex and, therefore we resort to suboptimal solutions. Two methods are introduced, namely, the higher order SVD (HOSVD) based design and the sequential low rank unimodular approximation based design. The former approach exploits the truncated HOSVD of the equivalent channel while the latter approach approximates optimal unconstrained solutions by low rank unimodular approximations. Simulation results show that when the mmWave channel model is used, both approaches outperform the extension of the state of the art compressed sensing based algorithm to the multi-carrier case.

In this letter, we consider a class of generalized multivariate Huber (GMH) loss functions. Our goal is parameter estimation in linear models contaminated by non-Gaussian noise. We assume access to a secondary dataset of independent noise realizations, and we use these data to fit a convex GMH function that will then lead to efficient parameter estimation. Our framework includes the classical weighted least squares and Huber's function as special cases. We demonstrate its advantages in heavy-tailed noise distributions.

We consider the joint estimation of structured covariance matrices. We assume the structure is unknown and perform the estimation using heterogeneous training sets. More precisely, we are given groups of measurements coming from centered normal populations with different covariance matrices. Assuming that all these covariance matrices span a low dimensional affine subspace in the space of symmetric matrices, our aim is to determine this structure. It is then utilized to improve the covariance estimation. We provide an algorithm discovering and exploring the underlying covariance structure and analyze its error bounds. Numerical simulations are presented to illustrate the performance benefits of the proposed algorithm.

This paper analyzes the performance of Tyler's M-estimator of the scatter matrix in elliptical populations. We focus on non-asymptotic performance analysis of Tyler's estimator. Given n samples of dimension p2/(1-c2)2n with high probability, where c is the coherence coefficient of the properly scaled estimator. Under additional group symmetry conditions we improve the obtained bound, utilizing the inherent sparsity properties of group symmetry.

We develop the General Method of Moments (GMM) Approach for estimating the covariance matrices of non-Gaussian distributions with convex structure. The GMM turns out to be a non-convex optimization problem, thus making the addition of prior knowledge in form of convex structure constraints cumbersome. We propose a different approach to this estimator and show that the Tyler's estimator can be obtained as a solution of a convexly relaxed GMM problem, thus making the imposition of convex constraints easier. This new framework provides consistent solutions which outperform the standard projection methods. As an application of this method we consider Gaussian Compound samples with Toeplitz and banded covariance matrices. We provide synthetic numerical data and demonstrate the performance advantages of our method.

We consider estimation of a deterministic unknown parameter vector in a linear model with non-Gaussian noise. In the Gaussian case, dimensionality reduction via a linear matched filter provides a simple low dimensional sufficient statistic which can be easily communicated and/or stored for future inference. Such a statistic is usually unknown in the general non-Gaussian case. Instead, we propose a hybrid matched filter coupled with a randomized compressed sensing procedure, which together create a low dimensional statistic. We also derive a complementary algorithm for robust reconstruction given this statistic. Our recovery method is based on the fast iterative shrinkage and thresholding algorithm which is used for outlier rejection given the compressed data. We demonstrate the advantages of the proposed framework using synthetic simulations.

We consider distributed estimation of the inverse covariance matrix, also called the concentration matrix, in Gaussian graphical models. Traditional centralized estimation often requires iterative and expensive global inference and is therefore difficult in large distributed networks. In this paper, we propose a general framework for distributed estimation based on a maximum marginal likelihood (MML) approach. Each node independently computes a local estimate by maximizing a marginal likelihood defined with respect to data collected from its local neighborhood. Due to the non-convexity of the MML problem, we derive and consider solving a convex relaxation. The local estimates are then combined into a global estimate without the need for iterative message-passing between neighborhoods. We prove that this relaxed MML estimator is asymptotically consistent. Through numerical experiments on several synthetic and real-world data sets, we demonstrate that the two-hop version of the proposed estimator is significantly better than the one-hop version, and nearly closes the gap to the centralized maximum likelihood estimator in many situations.

Semi-definite relaxation (SDR) is a popular technique for Multi-Input Multi-Output (MIMO) detection. For Binary Phase-Shift Keying (BPSK) and Quadratic Phase-Shift Keying (QPSK), it has been found that SDR can provide a near-optimal Bit Error rate (BER) performance in a Gaussian channel. However if the noise in the channel deviates from the Gaussian model, as it does in many real wireless channels, BER performance drops considerably. In this paper we show that SDR can be applied for detection in a non-Gaussian channel using Huber's M-estimation method for robust regression.

The autofocus problem in synthetic aperture radar imaging amounts to estimating unknown phase errors caused by unknown platform or target motion. At the heart of three state-of-the-art autofocus algorithms, namely, phase gradient autofocus, multichannel autofocus (MCA), and Fourier-domain multichannel autofocus (FMCA), is the solution of a constant modulus quadratic program (CMQP). Currently, these algorithms solve a CMQP by using an eigenvalue relaxation approach. We propose an alternative relaxation approach based on semidefinite programming, which has recently attracted considerable attention in other signal processing problems. Experimental results show that our proposed methods provide promising performance improvements for MCA and FMCA through an increase in computational complexity.

We introduce an efficient algorithm for performing distributed principal component analysis (PCA) on directed Gaussian graphical models. By exploiting structured sparsity in the Cholesky factor of the inverse covariance (concentration) matrix, our proposed DDPCA algorithm computes global principal subspace estimation through local computation and message passing. We show significant performance and computation/communication advantages of DDPCA for online principal subspace estimation and distributed anomaly detection in real-world computer networks.

Autofocus algorithms are used to restore images in nonideal synthetic aperture radar imaging systems. In this paper, we propose a bilinear parametric model for the unknown image and the nuisance phase parameters and derive an efficient maximum-likelihood autofocus (MLA) algorithm. In the special case of a simple image model and a narrow range of look angles, MLA coincides with the successful multichannel autofocus (MCA). MLA can be interpreted as a generalization of MCA to a larger class of models with a larger range of look angles. We analyze its advantages over previous extensions of MCA in terms of identifiability conditions and noise sensitivity. As a byproduct, we also propose numerical approximations to the difficult constant modulus quadratic program that lies at the core of these algorithms. We demonstrate the superior performance of our proposed methods using computer simulations in both the correct and mismatched system models. MLA performs better than other methods, both in terms of the mean squared error and visual quality of the restored image.

Autofocus algorithms deal with image restoration in a nonideal synthetic aperture radar (SAR) imaging system. We propose a novel autofocus algorithm, denoted as MLA, that is based on maximum likelihood estimation. MLA belongs to a class of autofocus algorithms that rely on a known low-return region in the underlying image. We find conditions under which MLA is equivalent to previous methods belonging to the same class. Simulation results show that when compared to previous methods, MLA performs better both in terms of visual quality of the restored image and mean square error (MSE) of the estimated unknown parameters.

The scalar shrinkage-thresholding operator is a key ingredient in variable selection algorithms arising in wavelet denoising, JPEG2000 image compression and predictive analysis of gene microarray data. In these applications, the decision to select a scalar variable is given as the solution to a scalar sparsity penalized quadratic optimization. In some other applications, one seeks to select multidimensional variables. In this work, we present a natural multidimensional extension of the scalar shrinkage thresholding operator. Similarly to the scalar case, the threshold is determined by the minimization of a convex quadratic form plus an Euclidean norm penalty, however, here the optimization is performed over a domain of dimensionN≥ 1. The solution to this convex optimization problem is called the multidimensional shrinkage threshold operator (MSTO). The MSTO reduces to the scalar case in the special case ofN=1. In the general case ofN>; 1 the optimal MSTO shrinkage can be found through a simple convex line search. We give an efficient algorithm for solving this line search and show that our method to evaluate the MSTO outperforms other state-of-the art optimization approaches. We present several illustrative applications of the MSTO in the context of Group LASSO penalized estimation.

Synthetic Aperture Radar (SAR) imaging can suffer from image focus degradation due to unknown platform or target motion. Autofocus algorithms use signal processing techniques to remove the undesired phase errors. The recently proposed multichannel autofocus models formulate the problem as the solution to Aejφ≈ 0, where A is a given matrix and φ are the unknown phases. Previous methods approximated ejf using the null vector of A. We propose to approximate ejφusing conic optimization and call this new autofocus algorithm Semidefinite Relaxation Autofocus (SDRA). Experimental results using a simulated SAR image shows that SDRA has promising performance advantages over existing autofocus methods.

We address covariance estimation under mean-squared loss in the Gaussian setting. Specifically, we consider shrinkage methods which are suitable for high dimensional problems with small number of samples (large p small n). First, we improve on the Ledoit-Wolf (LW) method by conditioning on a sufficient statistic via the Rao-Blackwell theorem, obtaining a new estimator RBLW whose mean-squared error dominates the LW under Gaussian model. Second, to further reduce the estimation error, we propose an iterative approach which approximates the clairvoyant shrinkage estimator. Convergence of this iterative method is proven and a closed form expression for the limit is determined, which is called the OAS estimator. Both of the proposed estimators have simple expressions and are easy to compute. Although the two methods are developed from different approaches, their structure is identical up to specific constants. The RBLW estimator provably dominates the LW method; and numerical simulations demonstrate that the OAS estimator performs even better, especially when n is much less than p.

We consider principal component analysis (PCA) in decomposable Gaussian graphical models. We exploit the prior information in these models in order to distribute its computation. For this purpose, we reformulate the problem in the sparse inverse covariance (concentration) domain and solve the global eigenvalue problem using a sequence of local eigenvalue problems in each of the cliques of the decomposable graph. We demonstrate the application of our methodology in the context of decentralized anomaly detection in the Abilene backbone network. Based on the topology of the network, we propose an approximate statistical graphical model and distribute the computation of PCA.

Wyner's wiretap channel is generalized to the case when the sender, the receiver and the eavesdropper have multiple antennas. We consider two cases: the deterministic case and the fading case. In the deterministic case, the channel matrices of the intended receiver and the eavesdropper are fixed and known to all the nodes. In the fading case, the channel matrices experience block fading and the sender has only the intended receiver's channel state information (CSI) and statistical knowledge of the eavesdropper's channel. For the deterministic case, a scheme based on the generalized-singular-value-decomposition (GSVD) of the channel matrices is proposed and shown to achieve the secrecy capacity in the high signal-to-noise-ratio (SNR) limit. When the intended receiver has only one antenna (MISO case) the secrecy-capacity is characterized for any SNR. Next, a suboptimal "artificial noise" based scheme is considered. Its performance is characterized and observed to be nearly optimal in the high SNR regime for the MISO case. This scheme extends naturally to the fading case and results are reported for the MISO case. For the independent Rayleigh fading distribution as we simultaneously increase the number of antennas at the sender and the eavesdropper, the secrecy capacity approaches zero if and only if the ratio of the number of eavesdropper antennas to transmitter antennas is at least two.

In this paper, we consider the optimization of the compound capacity in a rank one Ricean multiple input multiple output channel using partial channel state information at the transmitter side. We model the channel as a deterministic matrix within a known ellipsoid, and address the compound capacity defined as the maximum worst case mutual information in the set. We find that the optimal transmit strategy is always beamforming, and can be found using a simple one dimensional search. Similar results are derived for the worst case sum-rate of a multiple access channel with individual power constraints and a total power constraint. In this multiuser setting we assume equal array response at the receiver for all users. These results motivate the growing use of systems using simple beamforming transmit strategies

We are interested in studying different capacity formulations in linear vector Gaussian channels where the transmitter is only informed with the magnitude of the complex channel matrix coefficients and the receiver has perfect channel knowledge. Initially, we give the general expressions for the ergodic, compound, and outage capacities for our particular model of channel state information. Next, focusing on the compound formulation, we find that the optimal transmitter strategy consists in independent signaling through the transmit dimensions. Finally, we present a new result on the power allocation for the maximization of the outage mutual information.

We consider the problem of estimating an unknown deterministic parameter vector in a linear model with a Gaussian model matrix. The matrix has a known mean and independent rows of equal covariance matrix. Our problem formulation also allows for some known columns within this model matrix. We derive the maximum likelihood (ML) estimator associated with this problem and show that it can be found using a simple line-search over a unimodal function which can be efficiently evaluated. We then analyze its asymptotic performance using the Cramer Rao bound. Finally, we discuss the similarity between the ML, total least squares (TLS), and regularized TLS estimators

We consider the problem of estimating an unknown deterministic parameter vector in a linear model with a Gaussian model matrix. We derive the maximum likelihood (ML) estimator for this problem and show that it can be found using a simple line-search over a unimodal function that can be efficiently evaluated. We then discuss the similarity between the ML, the total least squares (TLS), the regularized TLS, and the expected least squares estimators.

Signal-to-noise ratio (SNR) estimation is considered for phase-shift keying communication systems in time-varying fading channels. Both data-aided (DA) estimation and nondata-aided (NDA) estimation are addressed. The time-varying fading channel is modeled as a polynomial-in-time. Inherent estimation accuracy limitations are examined via the Cramer-Rao lower bound, where it is shown that the effect of the channel's time variation on SNR estimation is negligible. A novel maximum-likelihood (ML) SNR estimator is derived for the time-varying channel model. In DA scenarios, where the estimator has a simple closed-form solution, the exact performance is evaluated both with correct and incorrect (i.e., mismatched) polynomial order. In NDA estimation, the unknown data symbols are modeled as random, and the marginal likelihood is used. The expectation-maximization algorithm is proposed to iteratively maximize this likelihood function. Simulation results show that the resulting estimator offers statistical efficiency over a wider range of scenarios than previously published methods.

We develop a computationally efficient approximation of the maximum likelihood (ML) detector for 16 quadrature amplitude modulation (16-QAM) in multiple-input multiple-output (MIMO) systems. The detector is based on a convex relaxation of the ML problem. The resulting optimization is a semidefinite program that can be solved in polynomial time with respect to the number of inputs in the system. Simulation results in a random MIMO system show that the proposed algorithm outperforms the conventional decorrelator detector by about 2.5 dB at high signal-to-noise ratios.

We consider the problem of designing a robust linear equalizer under channel uncertainties. Specifically, we propose a robust peak distortion (RPD) equalizer, in which we minimize the error probability of the worst-case sequence and the worst-case channel, in the uncertainty region. We show that the RPD equalizer can be found efficiently using standard convex optimization packages. We then demonstrate through simulations that under channel uncertainty the RPD equalizer outperforms traditional equalizers, and previously proposed robust equalizers.

We consider the problem of designing linear multiple input multiple output (MIMO) precoders for fixed receivers. We first derive a precoder that minimizes the average power subject to signal to interference plus noise ratio (SINR) constraints, and then derive a precoder that maximizes the worst case SINR subject to an average power constraint. We show that both problems can be solved using standard optimization packages. In addition, a more efficient solution based on Karush-Kuhn-Tucker (KKT) optimality conditions is presented which gives more insight into the problem. Our design promises equal SINR and fairness among the multiple outputs. Simulation results in a multiuser system show that the proposed precoders can significantly outperform existing linear precoders.

This work considers a high rate, multiple input multiple output (MIMO) systems using multiple codes, as well as channel coding and space time (ST) coding. The transmitter consists of a channel encoder followed by parallel linear dispersion codes (LDC) ST encoders using different spreading codes. The iterative receiver consists of a soft input and soft output (SISO) demodulator, followed by a SISO detector. Simulation results in realistic frequency selective third generation partnership project test cases reveal good performance even for high rate HSDPA services.

Data-Aided Signal-to-Noise-Ratio (SNR) estimation is considered for time selective fading channels whose time variation is described by a polynomial time model. The inherent estimation accuracy limitations associated with the problem are quantified via a Cramer-Rao Bound analysis. A maximum likelihood type class of estimators is proposed and its exact, non-asymptotic performance is computed. The standard, constant channel SNR estimator performance is determined in the presence of channel polynomial order mismatch. Simulations results are presented which verify the effectiveness of the technique as well as its performance advantage over previously proposed methods.

Non-data-aided (NDA) signal-to-noise-ratio (SNR) estimation is considered for binary phase shift keying systems where the data samples are governed by a normal mixture distribution. Inherent estimation accuracy limitations are examined via a simple, closed-form approximation to the associated Cramer-Rao bound which eliminates the need for numerical integration. The expectation-maximization algorithm is proposed to iteratively maximize the NDA likelihood function. Simulation results show that the resulting estimator offers statistical efficiency over a wider range of scenarios than previously published methods.