Recorrupted-to-Recorrupted (R2R) has emerged as a methodology for training deep networks for image restoration in a self-supervised manner from noisy measurement data alone, demonstrating equivalence in expectation to the supervised squared loss in the case of Gaussian noise. However, its effectiveness with non-Gaussian noise remains unexplored. In this paper, we propose Generalized R2R (GR2R), extending the R2R framework to handle a broader class of noise distribution as additive noise like log-Rayleigh and address the natural exponential family including Poisson, Gamma and Binomial noise distributions, which play a key role in many applications including low-photon imaging and synthetic aperture radar. We show that the GR2R loss is an unbiased estimator of the supervised loss and that the popular Stein’s unbiased risk estimator can be seen as a special case. A series of experiments with Gaussian, Poisson, and Gamma noise validate GR2R’s performance, showing its effectiveness compared to other self-supervised methods.

This paper introduces a plug-and-play algorithm for enhancing compressive spectral imaging (CSI) through the integration of both a quadratic envelope (QE) regularizer and a deep prior. Our method employs the QE-based regularizer to foster a low-rank structure in conjunction with deep priors, synergistically integrated within a Plug-and-Play (PnP) framework. The distinct advantage of our chosen QE-regularizer is its propensity for uncovering low-rank solutions devoid of bias, distinguishing it from the nuclear norm. Through this fusion of QE and deep priors, we harness the complementary strengths of both techniques, resulting in a mutually reinforcing effect for CSI.

Commercial sensors often suffer from overexposure in bright regions, leading to signal clipping and information loss because of saturation. Existing solutions involve either employing logarithmic irradiance response sensors or capturing multiple shots from different saturation levels. However, these approaches can be complex or rely on static scenes, limiting their effectiveness in fully addressing the saturation problem. A promising solution is the use of unsaturated sensors, also known as modulo cameras, which employ an array of self-reset pixels to wrap the signal when it reaches the saturation level. The resulting image exhibits a noisy and discontinuous shape, requiring an unwrapping algorithm to obtain a smooth and continuous representation of the scene. We propose a deep plug-and-play algorithm that combines model-based optimization with a deep denoiser. By leveraging the spatial correlation of the scene within the close solution of an unwrapping step, our approach successfully unwraps the continuous values while simultaneously reducing noise. Extensive evaluations show the superiority of our method compared to state-of-the-art unwrapping and unmodulo algorithms in terms of reconstruction quality.

Hadamard single-pixel imaging (HSI) is a promising sensing approach for acquiring spectral images in the near-infrared spectrum with high spatial resolution and fast recovery times due to the efficient invertible properties of the Hadamard matrix. The potential of the HSI system is diminished because of the large number of required measurements which implies long acquisition times. Recent advances proposed optimizing the HSI sensing matrix structure based on a superpixels map estimated from a side-information acquisition of the scene, reducing the number of required measurements. However, these matrix designs are detached from the recovery task, which falls on a sub-optimal strategy. In this work, we proposed an adaptive end-to-end sensing methodology for the HSI sensing matrix design based on deep superpixels estimation by coupling the sensing and recovery of the near-infrared spectral images. Experimental results show the superiority of the proposed sensing methodology compared with state-of-art sensing design schemes.

Deep learning End-to-End (E2E) approaches have emerged as alternative optical design models, which jointly train the optical parameters of the sensing protocol, and the parameters of the deep neural network to achieve a specific task. This E2E model is particularly useful in the design of coding optical systems to address relevant constraints of the coded aperture (CA) design. To name, recent works address the binary constraint by incorporating regularization functions in the E2E optimization problem to promote binary value entries. How-ever, they do not consider other important CA assembling properties as the transmittance level, which plays a crucial role in implementable setups. Therefore, this work proposes two transmittance regularizers that jointly induce binary en-tries and adjust the transmittance level to be incorporated in an E2E approach. In particular, one of the regularizers allows achieving an exact value of the transmittance level when required for specific applications.

This paper presents a recovery method for compressive spectral imaging (CSI) based on the training-data independent deep image prior approach, where the prior information of the image is learned through the weights and the structure of the neural network. Specifically, we propose an interpretable architecture inspired in the linear mixture model for spectral images, where the image is decomposed as the product between a basis matrix, known as endmembers, and a coefficient matrix, known as abundances. These matrices are learned as the weights and the features of the proposed network, respectively. Simulations and experiments show that the proposed recovery method outperforms the state-of-the-art CSI recovery methods, even against training-data dependent methods. Furthermore, the architecture structure inspired by the linear mixture model gives interpretability of some outputs that can be useful for subsequent high-level image processing.

This paper presents an end-to-end (E2E) deep learning approach for the fusion of the data from two compressive spectral imaging systems, where a single neural network is developed to simultaneously optimize the sensing matrices and the decoder operator. The proposed E2E method models the sensing operator of the systems to fuse as optical layers, where the learnable parameters are the coded apertures of these CSI systems. These optical layers are then concatenated to an inspired unrolled deep neural network, where after training, these sensing matrices remain non-trainable along the optimization stages. Finally, a loss function is proposed. Simulation results show an improvement of the proposed coupled method compared with previous work and an enhancement due to the training of the sensing matrices and the proposed loss function.

Single-pixel camera (SPC) captures encoded projections of the scene in a unique detector such that the number of compressive projections is lower than the size of the image. Traditionally, classification is not performed in the compressive domain because it is necessary to recover the underlying image before to classification. Based on the success of Deep learning (DL) in classification approaches, this paper proposes to classify images using compressive measurements of SPC. Furthermore, the proposed DL approach designs the binary sensing matrix in the SPC to improve the classification accuracy. In particular, a whole neural network is trained to learn the SPC sensing matrix, in the first layer, and extracts features from the single-pixel compressive measurements. The proposed approach overcomes two approaches of the state-of-the-art in terms of classification accuracy.

Super-resolution phase retrieval is an inverse problem that appears in diffractive optical imaging (DOI) and consists in estimating a high-resolution image from low-resolution phaseless measurements. DOI has three diffraction zones where the data can be acquired, known as near, middle, and far fields. Recent works have studied super-resolution phase retrieval under a setup that records coded diffraction patterns at the near and far fields. However, the attainable resolution of the image is mainly governed by the sensor characteristics, whose cost increases in proportion to the resolution. Also, these methodologies lack theoretical analysis. Hence, this work derives super-resolution models from low-resolution coded phaseless measurements at any diffraction zone that in contrast to prior contributions, the attainable resolution of the image is determined by the resolution of the coded aperture. For the proposed models, the existence of a unique solution (up to a global unimodular constant) is guaranteed with high probability, which can be increased by designing the coded aperture. Therefore, a strategy that designs the spatial distribution of the coded aperture is developed. Additionally, a super-resolution phase retrieval algorithm that minimizes a smoothed nonconvex least-squares objective function is proposed. The method first approximates the image by a spectral algorithm, which is then refined based upon a sequence of alternate steps. Simulation results show that the proposed algorithm overcomes state-of-the-art methods in reconstructing the high-resolution image. In addition, the reconstruction quality using designed coded apertures is higher than that of the non-designed ensembles.

Phase retrieval is a kind of ill-posed inverse problem, which is present in various applications, such as optics, astronomical imaging, and X-ray crystallography. Mathematically this inverse problem consists on recovering an unknown signal x ∈ Rn/Cnfrom a set of absolute square projections yk= |(ak, x)|2, k = 1,··· , m, where ak are the sampling vectors. However, the square absolute function is in general nonconvex and non-differentiable, which are desired properties in order to solve the problem, when traditional convex optimization algorithms are used. Therefore, this paper introduces a special differentiable function, known as smoothing function, in order to solve the phase retrieval problem by using the smoothing projected gradient (SPG) method. Moreover, to accelerate the convergence of this algorithm, this paper uses a nonlinear conjugate gradient method applied to the smoothing function as the search direction. Simulation results are provided to validate its efficiency on existing algorithms for phase retrieval. It is shown that compared with recently developed algorithms, the proposed method is able to accelerate the convergence.