ReLU-based hard thresholding (RHT) and ReLU-based hard thresholding pursuit (RHTP) are recently developed algorithms for non-negative sparse signal recovery. Although the restricted isometry property of order$3s$guarantees robust recovery when the restricted isometric constants satisfy$\delta _{3s} < 0.618$for RHT and$\delta _{3s} < 0.57735$for RHTP, numerical results show that RHTP consistently outperforms RHT, highlighting a gap between theoretical guarantees and observed performance. This work addresses this gap by deriving relaxed sufficient conditions for robust recovery based on$\delta _{2s}$. Specifically, we establish bounds for$\delta _{2s}$, with$\delta _{2s}< 0.379514$for RHTP and$\delta _{2s}< 0.357282$for RHT, demonstrating the broader applicability of RHTP. These results strengthen the theoretical guarantees, bringing them closer to empirical observations and advancing the understanding of ReLU-based methods.

The Hard Thresholding Pursuit (HTP) algorithm is a widely recognized method for sparse signal recovery, supported by rigorous theoretical guarantees based on the restricted isometry constants$\delta _{3s}$and$\delta _{2s}$. However, the existing analysis framework has not been extended to lower-order restricted isometry conditions. In this work, we address this gap by establishing a novel recovery guarantee for HTP under a relaxed condition involving$\delta _{s+1}$. Specifically, we show that HTP achieves robust recovery whenever$\delta _{s+1} < \sqrt{1/37} \approx 0.1644$, which surpasses the IHT-based bound of$\delta _{s+1} < 0.1545$based on the optimal step-length 1. To our knowledge, this result provides the first lower-order restricted isometry guarantee for HTP, in terms of$\delta _{s+1}$.

Filter pruning method is an effective approach to eliminate redundant filters in convolutional neural networks. Most of the existing techniques rely on threshold parameters for filter selection and employ multi-shot pruning, resulting in increased computational overhead. In addition, many methods primarily optimize test accuracy while overlooking robustness to noise, which is critical in practical and safety-critical applications. Since Incoherent Unit Norm Tight Frames (IUNTFs) are inherently robust to noise, we propose a filter pruning technique that removes redundant filters such that the retained set approximates an IUNTF structure. This structure enhances noise robustness and promotes the extraction of diverse and complementary features. Unlike conventional approaches, the proposed framework requires no threshold tuning and adopts a one-shot pruning strategy. Experimental results on the CIFAR-10 dataset using VGG-16, ResNet-56, and GoogLeNet demonstrate that the proposed method achieves robustness under noise and improved accuracy compared to existing approaches while pruning 85.12%, 57%, and 59% of the parameters and 69.66%, 57%, and 58% of the FLOPs, respectively.

In this article, we introduce a deep unfolding framework for Tail-iterative soft thresholding algorithm (ISTA) and Tail-fast ISTA (FISTA), extending classical sparse recovery algorithms into learned architectures and improving upon existing unfolding techniques. By combining the interpretability of iterative solvers with the adaptability of model-based networks, our approach achieves efficient and robust recovery of sparse signals. Tail-based methods incorporate an iterative support estimation step, where the support and target estimations are refined alternately, providing a key advantage over traditional approaches. We integrate this into our architecture, enhancing both recovery performance and noise robustness. We compare the proposed methods against classical solvers, including FISTA and Tail-FISTA, as well as deep unfolding techniques, LISTA and DU-FISTA, across various sparsity levels, dynamic ranges (DRs), and both noiseless and noisy conditions. In noiseless cases, our methods achieve slightly lower performance than classical solvers but with significantly reduced computational costs. Under heavy noise and a high number of nonzero elements, where classical methods struggle, our learned approaches remain resilient and achieve improved recovery rates. To evaluate generalization, we also tested our methods on data generated with a perturbed sensing matrix. In this case, under noisy scenarios, our proposed methods outperform classical sparse recovery algorithms. The proposed framework is general and applies to any linear sparse recovery task in compressed sensing (CS), offering computational efficiency, robustness to noise, and adaptability to real-world data, showcasing the advantages of deep unfolding techniques with iterative support estimation.

Linear Inverse Problems (LIPs) with non-negative sparse constraints on the target signal are critical in numerous applications across various fields. Among the recently proposed methods dealing with LIPs, model-based deep learning methods, particularly deep unrolling, have gained popularity due to their interpretability, efficiency, and superior performance compared to traditional iterative methods. In this work, we propose a model-driven deep learning method by unrolling the recently developed ReLU-based Hard Thresholding (RHT) algorithm for non-negative sparse signal recovery. Specifically, we employ soft thresholding activation instead of hard thresholding in the unrolling of RHT, which enhances the model performance during backpropagation. The primary advantages of the proposed Learned ReLU-based Soft Thresholding (LRST) include interpretability and faster inference after the network is trained. Our numerical experiments demonstrate that the proposed LRST outperforms its classical counterparts, showcasing its potential for more effective non-negative sparse signal recovery.

Recovering a sparse target vector with reduced sparsity from a given observation vector is a major challenge in many applications. The well-known tail-minimization approaches tackle this challenge by minimizing the tail part of the target vector. Building upon this, recent development, the tail fast iterative soft thresholding algorithm (Tail-FISTA) formulates the tail-minimization problem as an unconstrained l1-minimization problem and solves it with the FISTA method. Motivated by Tail-FISTA and the tail-minimization approaches, in this paper, we propose a sparse signal recovery algorithm called the recursive-tail-FISTA (R-Tail-FISTA). We employ a two-step procedure for R-Tail-FISTA: 1. We consider the tail-minimization problem and formulate it as an unconstrained l1-minimization problem. 2. We solve it by using the Tail-FISTA approach. We demonstrate that the R-Tail-FISTA method performs better in terms of sparse signal recovery compared to state-of-the-art algorithms. Additionally, we demonstrate the superiority of R-Tail-FISTA by recursively applying the tail-minimization technique to the tail part of the target vector twice. Furthermore, we numerically show that the convergence rate is better for R-Tail-FISTA than that of Tail-FISTA.

The non-negative least squares (NNLS) aims at finding a non-negative approximation of a matrix system. Such an approximation has been realized in the literature via some iterative methods. Recently, unrolling algorithms have gained significant attention due to their superior approximation results compared to data-driven methods. With a view to obtaining a better approximation, in this work, we deal with the NNLS in a deep learning framework by unrolling the Proximal Gradient Descent Method (PGDM). In the proposed method (referred to as unrolled or learned PGDM), we project the points to the non-negative orthant, which naturally gives rise to the ReLU function in the network as a proximal operator. The network is trained end-to-end in a deep-learning framework. We demonstrate empirically that the Unrolled PGDM (UPGDM) performs better in signal recovery than its classical counterpart involving iterative PGDM.