Reconstructing hyperspectral images (HSIs) from coded measurements from coded aperture snapshot spectral imaging (CASSI) system is essential for capturing images that offer superior spectral resolution over traditional RGB images. Nevertheless, the reconstruction of HSIs is difficult because of the aliasing of spatial and spectral information in coded measurements. To address this issue, we explore the possibility of utilizing an RGB camera with a Bayer color filter array (CFA) as an alternative to the gray-scale camera for implementing CASSI, which we term Bayer-CASSI. We first formulate the mathematical model of Bayer-CASSI, and then we evaluate the performance of Bayer-CASSI in simulation data and real data captured by our self-built Bayer-CASSI optical system. Experiment results in both simulation and real data show the effectiveness of Bayer-CASSI.

Hyperspectral image (HSI) reconstruction from coded measurement is significant for acquiring images with higher spectral resolution than traditional RGB images. Current advanced neural networks have already shown impressive performance in some datasets like CAVE and KAIST. However, these networks rely on a large amount of simulated ground truth, measurement pairs. Unfortunately, in some scenarios, it is hard to obtain a sufficient high-quality HSI training set, resulting in low generalization ability. Although iterative algorithms show good generalization ability, they are limited by slow speed and low reconstruction quality. To address this challenge, in this paper, we propose a self-supervised learning framework, which can train and fine-tune networks using measurements without ground truth. Besides, we propose the spectral low-rank loss function that enables networks to learn the signal model of HSI. Finally, we train and fine-tune a representative deep unfolding network, GAP-net, using our proposed framework. Extensive simulation and real data results show that the proposed self-supervised framework is capable of achieving results competitive with those of supervised networks. Code is available at https://github.com/zjhe02/CASSI-SSL.

Conventional high-speed and spectral imaging systems are expensive and they usually consume a significant amount of memory and bandwidth to save and transmit the high-dimensional data. By contrast, snapshot compressive imaging (SCI), where multiple sequential frames are coded by different masks and then summed to a single measurement, is a promising idea to use a 2-dimensional camera to capture 3-dimensional scenes. In this paper, we consider the reconstruction problem in SCI, i.e., recovering a series of scenes from a compressed measurement. Specifically, the measurement and modulation masks are fed into our proposed network, dubbed BIdirectional Recurrent Neural networks with Adversarial Training (BIRNAT) to reconstruct the desired frames. BIRNAT employs a deep convolutional neural network with residual blocks and self-attention to reconstruct the first frame, based on which a bidirectional recurrent neural network is utilized to sequentially reconstruct the following frames. Moreover, we build an extended BIRNAT-color algorithm for color videos aiming at joint reconstruction and demosaicing. Extensive results on both video and spectral, simulation and real data from three SCI cameras demonstrate the superior performance of BIRNAT.

We consider using untrained neural networks to solve the reconstruction problem of snapshot compressive imaging (SCI), which uses a two-dimensional (2D) detector to capture a high-dimensional (usually 3D) data-cube in a compressed manner. Various SCI systems have been built in recent years to capture data such as high-speed videos, hyperspectral images, and the state-of-the-art reconstruction is obtained by the deep neural networks. However, most of these networks are trained in an end-to-end manner by a large amount of corpus with sometimes simulated ground truth, measurement pairs. In this paper, inspired by the untrained neural networks such as deep image priors (DIP) and deep decoders, we develop a framework by integrating DIP into the plug-and-play regime, leading to a self-supervised network for spectral SCI reconstruction. Extensive synthetic and real data results show that the proposed algorithm without training is capable of achieving competitive results to the training based networks. Furthermore, by integrating the proposed method with a pre-trained deep denoising prior, we have achieved state-of-the-art results. Our code is available at https://github.com/mengziyi64/CASSI-Self-Supervised.

We consider the inverse problem of coded aperture snapshot spectral imaging (CASSI), which captures the spatio-spectral data-cube using a snapshot 2D measurement and reconstructs the 3D hyperspectral images using algorithms. Recent advances of deep learning have boosted the image quality of the reconstructed hyperspectral images significantly, and this leads to an end-to-end real-time capture and reconstruction system. However, the network design for CASSI reconstruction is still at the incubation stage and usually an off-the-shelf network is employed and re-purposed. In this work, from a different perspective, inspired by the fact that most existing hyperspectral images are still in the visible bandwidth, we introduce the perceptual loss into the deep neural network for CASSI reconstruction. Extensive results on both simulation and real data demonstrate that with this small change, the reconstructed image quality can be improved dramatically using the same network.