We present a powerful solution to the problem of vector set classification, based on a novel goodness-of-fit metric, referred to as signal subspace matching (SSM). Unlike the existing solutions based on principal component analysis (PCA), this solution is eigendecomposition-free and dimension-selection-free, i.e., it does not require PCA nor the election of the subspace dimension, which is done implicitly. More importantly, it copes effectively with the challenging cases wherein the subspaces characterizing the classes are partially or fully overlapping. The SSM metric matches the subspaces characterizing the vector sets of the test and the classes by minimizing the distance between respective soft-projection matrices constructed from the vector sets. We prove the consistency of the solution for the high signal-to-noise-ratio limit, and also for the large-sample limit, conditioned on the noise being white. Experimental results, demonstrating the superiority of the SSM solution over the existing PCA-based solutions, especially in the challenging cases of overlapping subspaces, are included.

We present a novel and computationally simple solution to the problem of determining the number of exponentials in a given time-series, which is applicable to both white and colored noise. The solution is based on a novel and non-asymptotic goodness-of-fit metric, referred to as invariant-signal-subspace matching (ISSM). This metric is aimed at matching pairs of signal-subspaces, created by exploiting the shift-invariance property of the Hankel data-matrix. A pair of such subspaces, together with their corresponding projection matrices, is created for every hypothesized number of exponentials, and the number of exponentials is then determined as that for which the distance between the pair of projection matrices is minimized. We prove the consistency of this criterion for the high signal-to-noise-ratio limit and also prove it for the large-sample limit, conditioned on the noise being white. We also extend this criterion to include multiple pairs of invariant subspace, readily created from the Hankel data-matrix, thus enabling to improve its performance at a slight increase in its computational load. Simulation results, demonstrating the superior performance of the solution over the existing solutions, for both colored and white noise, are included.

We present a novel convex-optimization-based solution to blind linear multiuser detection in direct-sequence code division multiple access systems. The solution is based on a convex low-rank approximation of the linearly constrained constant modulus cost function, thus guaranteeing its global minimization. Further, it can be cast as a semidefinite program, implying that it can be solved using interior-point techniques with polynomial time complexity. The solution is parameter free and is shown to be superior to existing solutions in terms of output signal-to-interference-plus-noise ratio and bit error rate, especially for a small number of samples.

Compressed sensing (CS) is a signal processing framework for efficiently reconstructing a signal from a small number of measurements, obtained by linear projections of the signal. Block-based CS is a lightweight CS approach that is mostly suitable for processing very high-dimensional images and videos: it operates on local patches, employs a low-complexity reconstruction operator and requires significantly less memory to store the sensing matrix. In this paper we present a deep learning approach for block-based CS, in which a fully-connected network performs both the block-based linear sensing and non-linear reconstruction stages. During the training phase, the sensing matrix and the non-linear reconstruction operator are jointly optimized, and the proposed approach out-performs state-of-the-art both in terms of reconstruction quality and computation time. For example, at a 25% sensing rate the average PSNR advantage is 0.77dB and computation time is over 200-times faster.

We consider the problem of greedy sparse approximation in the presence of noise, given a-priori knowledge of the sparse coefficients' covariance and mean. The proposed Covariance-Assisted Matching Pursuit (CAMP) combines the a-priori knowledge by leveraging the Gauss-Markov theorem, and provides significantly better performance than the classical Orthogonal Matching Pursuit (OMP). This improvement is achieved by solving in each matching pursuit stage a weighted least-squares problem that provides the best linear unbiased estimator of the sparse representation. The covariance and mean of the sparse representation coefficients can be estimated by a simple procedure from training data, and the advantage of the proposed approach is demonstrated for the tasks of denoising and inpainting 150,000 patches from natural images.

We present a linear-time subspace clustering approach that combines sparse representations and bipartite graph modeling. The signals are modeled as drawn from a union of low-dimensional subspaces, and each signal is represented by a sparse combination of basis elements, termed atoms, which form the columns of a dictionary matrix. The sparse representation coefficients are arranged in a sparse affinity matrix, which defines a bipartite graph of two disjoint sets: 1) atoms and 2) signals. Subspace clustering is obtained by applying low-complexity spectral bipartite graph clustering that exploits the small number of atoms for complexity reduction. The complexity of the proposed approach is linear in the number of signals, thus it can rapidly cluster very large data collections. Performance evaluation of face clustering and temporal video segmentation demonstrates comparable clustering accuracies to state-of-the-art at a significantly lower computational load.

We present a probabilistic subspace clustering approach that is capable of rapidly clustering very large signal collections. Each signal is represented by a sparse combination of basis elements (atoms), which form the columns of a dictionary matrix. The set of sparse representations is utilized to derive the co-occurrences matrix of atoms and signals, which is modeled as emerging from a mixture model. The components of the mixture model are obtained via a non-negative matrix factorization (NNMF) of the co-occurrences matrix, and the subspace of each signal is estimated according to a maximum-likelihood (ML) criterion. Performance evaluation demonstrate comparable clustering accuracies to state-of-the-art at a fraction of the computational load.

We propose the audio inpainting framework that recovers portions of audio data distorted due to impairments such as impulsive noise, clipping, and packet loss. In this framework, the distorted data are treated as missing and their location is assumed to be known. The signal is decomposed into overlapping time-domain frames and the restoration problem is then formulated as an inverse problem per audio frame. Sparse representation modeling is employed per frame, and each inverse problem is solved using the Orthogonal Matching Pursuit algorithm together with a discrete cosine or a Gabor dictionary. The Signal-to-Noise Ratio performance of this algorithm is shown to be comparable or better than state-of-the-art methods when blocks of samples of variable durations are missing. We also demonstrate that the size of the block of missing samples, rather than the overall number of missing samples, is a crucial parameter for high quality signal restoration. We further introduce a constrained Matching Pursuit approach for the special case of audio declipping that exploits the sign pattern of clipped audio samples and their maximal absolute value, as well as allowing the user to specify the maximum amplitude of the signal. This approach is shown to outperform state-of-the-art and commercially available methods for audio declipping in terms of Signal-to-Noise Ratio.

We present a novel sparse representation based approach for the restoration of clipped audio signals. In the proposed approach, the clipped signal is decomposed into overlapping frames and the declipping problem is formulated as an inverse problem, per audio frame. This problem is further solved by a constrained matching pursuit algorithm, that exploits the sign pattern of the clipped samples and their maximal absolute value. Performance evaluation with a collection of music and speech signals demonstrate superior results compared to existing algorithms, over a wide range of clipping levels.

We present a novel weighted approach for shrinkage functions learning in image denoising. The proposed approach optimizes the shape of the shrinkage functions and maximizes denoising performance by emphasizing the contribution of sparse overcomplete representation components. In contrast to previous work, we apply the weights in the overcomplete domain and formulate the restored image as a weighted combination of the post-shrinkage overcomplete representations. We further utilize this formulation in an offline Least Squares learning stage of the shrinkage functions, thus adapting their shape to the weighting process. The denoised image is reconstructed with the learned weighted shrinkage functions. Computer simulations demonstrate superior shrinkage-based denoising performance.