We investigate the two primary categories of structured recovery problems, namely Compressed Sensing (CS) and Low Rank Recovery (LRR). Our focus is on the performance analysis of their two tightest convex relaxation based heuristics, the so-called$\ell _{1}$and the nuclear norm ($\ell _{1}^{*}$) minimizations. We examine two standard types of phase transitions (PTs): 1) general PT, obtained by enforcing sparsity as a fundamental form of structuring, and 2) nonnegative PT, achieved by imposing nonnegativity as an additional form of structuring alongside sparsity. We establish explicit relations between the CS and LRR PTs. Our analysis reveals that the nonnegative PT essentially interpolates between the general and the binary CS PT, in a manner that can be explicitly characterized. Quite surprisingly, although the phase transitions themselves admit fairly complicated mathematical formulations, their relations can be expressed in a very neat and elegant way. This ultimately allows to quickly assess and compare the effects additional presence/absence of the nonnegativity has on$\ell _{1}$and$\ell _{1}^{*}$.

In this paper, we establish a rigorous connection between Causal inference (C-inf) and the low-rank recovery (LRR). Using Random Duality Theory (RDT) and novel mathematical strategies related to free probability theory, we obtain the exact explicit typical (and achievable) worst case phase transitions (PT). These PT precisely separate scenarios where causal inference via LRR is possible from those where it is not. We supplement our mathematical analysis with numerical experiments that confirm the theoretical predictions of PT phenomena, and further show that the two closely match for fairly small sample sizes. We obtain simple closed form representations for the resulting PTs, which highlight direct relations between the low rankness of the target C-inf matrix and the time of the treatment. Hence, our results can be used to determine the range of C-inf’s typical applicability.

In this paper we revisit two groups of the most famous structured recovery problems, the compressed sensing (CS) and the low-rank recovery (LRR). Our particular focus is on two tightest convex relaxation based heuristics, the so-called ℓ1and the nuclear norm $\left({l_1^ * }\right)$ minimizations, and their accompanying phase transitions (PT) phenomena. For concreteness, we consider two standard types of PTs, the first obtained by imposing sparsity and the second obtained by additionally imposing nonnegativity. We establish an explicit relation between these two types of PTs in CS, and the two corresponding types of PTs in the LRR. We also uncover a connection between PTs in CS and LRR. Our results make it possible to compare in a mathematically rigorous way both the level of heuristics’ successes and the effects that different types of structuring can have on the recovery process. Quite surprisingly, although the phase transitions themselves admit fairly complicated mathematical formulations, their relations can be expressed in a very neat and elegant way which allows for a quick assessment of the heuristic approximation quality.

Recently, theoretically analyzed the success of a polynomial ℓ1-optimization algorithm in solving an under-determined system of linear equations. In a large dimensional and statistical context proved that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that ℓ1-optimization succeeds in solving the system. In this paper, we consider an alternative performance analysis of ℓ1-optimization and demonstrate that the proportionality constants it provides in certain cases match or improve on the best currently known ones from.

One of the most basic problems in compressed sensing is solving an under-determined system of linear equations. Although this problem seems rather hard certain ℓ1-optimization algorithm appears to be very successful in solving it. The recent work of rigorously proved (in a large dimensional and statistical context) that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that ℓ1-optimization algorithm succeeds in solving the system. In more recent papers we considered the setup of the so-called block-sparse unknown vectors. In a large dimensional and statistical context, we determined sharp lower bounds on the values of allowable sparsity for any given number (proportional to the length of the unknown vector) of equations such that an ℓ2/ℓ1-optimization algorithm succeeds in solving the system. The results established in assumed a fairly large block-length of the block-sparse vectors. In this paper we consider the block-length to be a parameter of the system. Consequently, we then establish sharp lower bounds on the values of the allowable block-sparsity as functions of the block-length.

Recently, theoretically analyzed the success of a polynomial ℓ1-optimization algorithm in solving an under-determined system of linear equations. In a large dimensional and statistical context proved that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that ℓ1-optimization succeeds in solving the system. In this paper, we consider an alternative performance analysis of ℓ1-optimization and demonstrate that linear sparsity is recoverable for a large class of almost deterministic measurement matrices.

Recently, theoretically analyzed the success of a polynomial ℓ1-optimization algorithm in solving an under-determined system of linear equations. In a large dimensional and statistical context proved that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that ℓ1-optimization succeeds in solving the system. In this paper, we consider the same problem while additionally assuming that the locations of a fraction of non-zero elements are a priori known. We provide a performance analysis of a modified ℓ1-optimization. As expected, the obtained recoverable sparsity proportionality constants improve on the equivalent ones that can be obtained if no information about the non-zero elements location is available. This effectively means that, for certain sparsity levels where ℓ1-optimization is unsuccessful, instead of solving exactly an under-determined system of linear equations one may be interested in searching for approximate algorithms that only recover location of the fraction of the non-zero part of the unknown vector.

Recently, theoretically analyzed the success of a polynomial ℓ1optimization algorithm in solving an under-determined system of linear equations. In a large dimensional and statistical context proved that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that ℓ1optimization succeeds in solving the system. In this paper, we consider the same problem while additionally assuming that all non-zero elements elements are equal to each other. We provide a performance analysis of a slightly modified ℓ1optimization. As expected, the obtained recoverable sparsity proportionality constants improve on the equivalent ones that can be obtained if no information about the non-zero elements is available. In addition, we conducted a sequence of numerical experiments and observed that the obtained theoretical proportionality constants are in a solid agreement with the ones obtained experimentally.

It is well known that compressed sensing problems reduce to solving large under-determined systems of equations. If we choose the elements of the compressed measurement matrix according to some appropriate probability distribution and if the signal is sparse enough then the l1optimization can recover it with overwhelming probability (see, e.g. [4], [6], [7]). In fact, [4], [6], [7] establish (in a statistical context) that if the number of measurements is proportional to the length of the signal then there is a sparsity of the unknown signal proportional to its length for which the success of the l1optimization is guaranteed. In this paper we introduce a novel, very simple technique for proving this fact. Furthermore, in addition to being very simple the new technique provides very good values for proportionality constants. In some cases, the presented analysis, although very simple, provides the best currently known values for the proportionality constants.

It has been known for a while that l1-norm relaxation can in certain cases solve an under-determined system of linear equations. Recently, [5, 10] proved (in a large dimensional and statistical context) that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that l1-norm relaxation succeeds in solving the system. In this paper we determine sharp lower bounds on the values of allowable sparsity for any given number (proportional to the length of the unknown vector) of equations for the case of the so-called block-sparse unknown vectors considered in [25].

It has been known for a while that lscr1-norm relaxation can in certain cases solve an under-determined system of linear equations. Recently, proved (in a large dimensional and statistical context) that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that lscr1-norm relaxation succeeds in solving the system. In this paper we consider a modification of this standard setup, namely the case of so-called approximately block-sparse unknown vectors. We determine sharp lower bounds on the values of allowable approximate block-sparsity for any given number (proportional to the length of the unknown vector) of equations. Obtained lower bounds on the allowable sparsity are as expected functions of a parameter used to describe how close the approximately block-sparse unknown vectors are to the ideally block-sparse ones.

It is well known that compressed sensing problems reduce to solving large under-determined systems of equations. If we choose the elements of the compressed measurement matrix according to some appropriate probability distribution and if the signal is sparse enough then the lscr1-optimization can recover it with overwhelming probability. In fact, establish (in a statistical context) that if the number of measurements is proportional to the length of the signal then there is a sparsity of the unknown signal proportional to its length for which the success of the lscr1-optimization is guaranteed. In this paper we consider a modification of this standard setup, namely the case of the so-called approximately sparse unknown signals. We determine sharp lower bounds on the values of allowable approximate sparsity for any given number (proportional to the length of the unknown signal) of measurements. We introduce a novel, very simple technique which provides very good values for proportionality constants.

It is well known that compressed sensing problems reduce to solving large under-determined systems of equations. To assure that the problem is well defined, i.e., that the solution is unique the vector of unknowns is of course assumed to be sparse. Nonetheless, even when the solution is unique, finding it in general may be computationally difficult. However, starting with the seminal work of [2], it has been shown that linear programming techniques, obtained from an l1-norm relaxation of the original non-convex problem, can provably find the unknown vector in certain instances. In particular, using a certain restricted isometry property, [2] shows that for measurement matrices chosen from a random Gaussian ensemble, l1optimization can find the correct solution with overwhelming probability even when the number of non-zero entries of the unknown vector is proportional to the number of measurements (and the total number of unknowns). The subsequent paper [1] uses results on neighborly polytopes from [5] to give a “sharp” bound on what this proportionality should be in the Gaussian case. In the current paper, we observe that what matters is not so much the distribution from which the entries of the measurement matrix A are drawn, but rather the statistics of the null-space of A. Using this observation, we provide an alternative proof of the main result of [2] by analyzing matrices whose null-space is isotropic (of which i.i.d. Gaussian ensembles are a special case).

The demand for high data rate reliable communications poses great challenges to the next generation wireless systems in highly dynamic mobile environments. In this paper, we investigate the joint maximum-likelihood (ML) channel estimation and signal detection problem for single-input multiple-output (SIMO) wireless systems with general modulation constellations and propose an efficient sequential decoder for finding the exact joint ML solution. Unlike other known methods, the new decoder can even efficiently find the joint ML solution under high spectral efficiency non-constant modulus modulation constellations. In particular, the new algorithm does not need such preprocessing steps as Cholesky or QR decomposition in the traditional sphere decoders for joint ML channel estimation and data detection. The elimination of such preprocessing not only reduces the number of floating point computations, but also will potentially lead to smaller size and power consumption in VLSI implementations while providing better numerical stability.

It is well known that compressed sensing problems reduce to solving large under-determined systems of equations. If we choose the compressed measurement matrix according to some appropriate distribution and the signal is sparse enough the l1optimization can exactly recover the ideally sparse signal with overwhelming probability [2], [1]. In the current paper, we will consider the case of the so-called approximately sparse signals. These signals are a generalized version of the ideally sparse signals. Letting the zero valued components of the ideally sparse signals to take the values of certain small magnitude one can construct the approximately sparse signals. Using a different but simple proof technique we show that the claims similar to those of [2] and [1] related to the proportionality of the number of large components of the signals to the number of measurements, hold for approximately sparse signals as well. Furthermore, using the same technique we compute the explicit values of what this proportionality can be if the compressed measurement matrix A has a rotationally invariant distribution of the null-space. We also give the quantitative tradeoff between the signal sparsity and the recovery robustness of the l1minimization. As it will turn out in an asymptotic case of the number of measurements the threshold result of [1] corresponds to a special case of our result.

Fast fading wireless environments pose a great challenge for achieving high spectral efficiency in next generation wireless systems. Joint maximum-likelihood (ML) channel estimation and signal detection is of great theoretical and practical interest, especially for multiple-input multiple-output(MIMO) systems where the multiple channel coefficients need to be estimated. However, this is a hard combinatorial optimization problem, for which obtaining efficient exact algorithms has been elusive for the general MIMO systems. In this paper, we propose an efficient branch-estimate-bound non-coherent optimization framework which provably achieves the exact ML joint channel estimation and data detection for general MIMO systems. Numerical results indicate that the exact joint ML method can achieve substantial performance improvements over suboptimal methods including iterative channel estimation and signal detection. We also derive analytical bounds on the computational complexity of the new exact joint ML method and show that its average complexity approaches a constant times the length of the coherence time, as the SNR approaches infinity.

In multi-antenna communication systems, channel information is often not known at the receiver. To fully exploit bandwidth resources of the system and ensure practical feasibility of the receiver, channel parameters are often estimated blindly and then employed in the design of signal detection algorithms. Instead of separating channel estimation from signal detection, in this paper we focus on the joint channel estimation and signal detection problem in a single-input multiple-output (SIMO) system. It is well known that finding solution to this optimization requires solving an integer maximization of a quadratic form and is, in general, an NP hard problem. To solve it, we propose an approximate algorithm based on the semi-definite program (SDP) relaxation. We derive a bound on the pairwise probability of error (PEP) of the proposed algorithm and show that, the algorithm achieves the same diversity as the exact maximum-likelihood (ML) decoder. The computed PEP implies that, over a wide range of system parameters, the proposed algorithm requires moderate increase in the signal-to-noise ratio (SNR) in order to achieve performance comparable to that of the ML decoder but with often significantly lower complexity.

In this paper we consider the problem of the optimal quantum unambiguous detection between two mixed quantum states. More specifically, we consider two mixed quantum states of rank 2 which lie in a Hilbert space of dimension 4. Using duality theory we explicitly characterize the optimal measurement operators. Furthermore, as a by-product of our framework we obtain a closed form solution of unambiguous discrimination between a pure and a mixed quantum state.

In multi-antenna communication systems, channel information is often not known at the receiver. To fully exploit the bandwidth resources of the system and ensure the practical feasibility of the receiver, the channel parameters are often estimated and then employed in the design of signal detection algorithms. However, sometimes communication can occur in the environment where learning the channel coefficients becomes infeasible. In this paper we consider the problem of maximum-likelihood (ML)-detection in single-input multiple-output (SIMO) systems when the channel information is completely unavailable at the receiver and when employed signalling at the transmitter is q-PSK. It is well known that finding the solution to this optimization requires solving an integer maximization of a quadratic form and is, in general, an NP hard problem. To solve it, we propose an approximate algorithm based on the semi-definite program (SDP) relaxation. We derive a bound on the pairwise probability of error (PEP) of the proposed algorithm and show that, the algorithm achieves the same diversity as the exact maximum-likelihood (ML) decoder. Furthermore, we prove that in the limit of large system dimension this bound differs from the corresponding one in the exact ML case by at most 3.92 dB if the transmitted symbols are from 2 or 4-PSK constellations and by at most 2.55 dB if the transmitted symbols are from 8-PSK constellation. This suggests that the proposed algorithm requires moderate increase in the signal-to-noise ratio (SNR) in order to achieve performance comparable to that of the ML decoder but with often significantly lower complexity.

In many communication applications, maximum-likelihood decoding reduces to solving an integer least-squares problem which is NP hard in the worst-case. On the other hand, it has recently been shown that, over a wide range of dimensions and SNR, the sphere decoder can be used to find the exact solution with an expected complexity that is roughly cubic in the dimension of the problem. However, the computational complexity becomes prohibitive if the SNR is too low and/or if the dimension of the problem is too large. In earlier work, we targeted these two regimes attempting to find faster algorithms by pruning the search tree beyond what is done in the standard sphere decoder. The search tree is pruned by computing lower bounds on the possible optimal solution as we proceed to go down the tree. A trade-off between the computational complexity required to compute the lower bound and the size of the pruned tree is readily observed: the more effort we spend in computing a tight lower bound, the more branches that can be eliminated in the tree. Thus, even though it is possible to prune the search tree (and hence the number of points visited) by several orders of magnitude, this may be offset by the computations required to perform the pruning. In this paper, we propose a computationally efficient lower bound which requires solving a single semi-definite program (SDP) at the top of the search tree; the solution to the SDP is then used to deduce the lower bounds on the optimal solution on all levels of the search tree. Simulation results indicate significant improvement in the computational complexity of the proposed algorithm over the standard sphere decoding

The sum rate capacity of the multi-antenna Gaussian broadcast channel has recently been computed. However, the search for computationally efficient practical schemes that achieve it is still in progress. When the channel state information is fully available at the transmitter, the dirty paper coding (DPC) technique is known to achieve the maximal throughput, but is computationally infeasible. In this paper, we analyze the asymptotic behavior of one of its alternatives - the recently suggested so-called vector perturbation technique. We show that for a square channel, where the number of users is large and equal to the number of transmit antennas, its sum rate approaches that of the DPC technique. More precisely, we show that at both low and high signal-to-noise ratio (SNR), the scheme under consideration is asymptotically optimal. Furthermore, we obtain similar results in the case where the number of users is much larger than the number of transmit antennas

The sum rate capacity of the multi-antenna broadcast channel has recently been computed. However, the search for efficient practical schemes that achieve it is still ongoing. In this paper, we focus on schemes with linear preprocessing of the transmitted data. We propose two criteria for the preceding matrix design: one maximizing the sum rate and the other maximizing the minimum rate among all users. The latter problem is shown to be quasiconvex and is solved exactly via a bisection method. In addition to preceding, we employ a signal scaling scheme that minimizes the average bit-error-rate (BER). The signal scaling scheme is posed as a convex optimization problem, and thus can be solved exactly via efficient interior-point methods. In terms of the achievable sum rate, the proposed technique significantly outperforms traditional channel inversion methods, while having comparable (in fact, often superior) BER performance

In many communications applications, maximum-likelihood decoding reduces to solving an integer least-squares problem which is NP hard in the worst-case. However, as has recently been shown, over a wide range of dimensions and SNRs, the sphere decoder can be used to find the exact solution with an expected complexity that is roughly cubic in the dimension of the problem. However, the computational complexity becomes prohibitive if the SNR is too low and/or if the dimension of the problem is too large. We target these two regimes and attempt to find faster algorithms by pruning the search tree beyond what is done in the standard sphere decoder. The search tree is pruned by computing lower bounds on the possible optimal solution as we proceed down the tree. We observe a trade-off between the computational complexity required to compute the lower bound and the size of the pruned tree: the more effort spent computing a tight lower bound, the more branches that can be eliminated in the tree. Thus, even though it is possible to prune the search tree (and hence the number of points visited) by several orders of magnitude, this may be offset by the computations required to perform the pruning. All of which suggests the need for computationally-efficient tight lower bounds. We present three different lower bounds (based on spherical-relaxation, polytope-relaxation and duality), simulate their performances and discuss their relative merits.

The sum rate capacity of the multi-antenna broadcast channel has recently been computed. However, the search for efficient practical schemes that achieve it is still ongoing. In this paper, we focus on schemes with linear preprocessing of the transmitted data. We propose two criteria for precoding matrix design, one maximizing the sum rate and the other maximizing the minimum rate among all users. The latter problem is shown to be quasiconvex and is solved exactly via the bisection method. In addition to precoding, we employ a signal scaling scheme that minimizes average bit-error-rate (BER). The signal scaling scheme is posed as a convex optimization problem, and thus can be solved exactly via efficient interior-point methods. In terms of achievable sum rate, the proposed technique significantly outperforms traditional channel inversion methods, while having comparable (in fact, often superior) BER performance.