Images taken in a low light condition with the presence of camera shake suffer from motion blur and photon shot noise. While state-of-the-art image restoration networks show promising results, they are largely limited to well-illuminated scenes and their performance drops significantly when photon shot noise is strong. In this paper, we propose a new blur estimation technique customized for photon-limited conditions. The proposed method employs a gradient-based backpropagation method to estimate the blur kernel. By modeling the blur kernel using a low-dimensional representation with the key points on the motion trajectory, we significantly reduce the search space and improve the regularity of the kernel estimation problem. When plugged into an iterative framework, our novel low-dimensional representation provides improved kernel estimates and hence significantly better deconvolution performance when compared to end-to-end trained neural networks. The source code and pretrained mdoels are available at https://github.com/sanghviyashiitb/structured-kernel-cvpr23

Image deblurring in a photon-limited condition is ubiquitous in a variety of low-light applications such as photography, microscopy and astronomy. However, the presence of photon shot noise due to a low-illumination and/or short exposure makes the deblurring task substantially more challenging than conventional deblurring. In this paper we present an algorithm unrolling approach that unrolls a Plug-and-Play algorithm using a fixed-iteration network. By changing the conventional two-variable splitting formulation of Plug-and-Play to an alternate three-variable splitting, we obtain a differentiable and end-to-end trainable network. Our algorithm outperforms existing methods for 1dB across different illuminations. We also overcome the difficulty of acquiring real motion blur kernels at low-light by presenting a photon-limited motion deblurring image dataset.

A 3D point cloud is typically constructed from depth measurements acquired by sensors at one or more viewpoints. The measurements suffer from both quantization and noise corruption. To improve quality, previous works denoise a point cloud a posteriori after projecting the imperfect depth data onto 3D space. Instead, we enhance depth measurements directly on the sensed images a priori, before synthesizing a 3D point cloud. By enhancing near the physical sensing process, we tailor our optimization to our depth formation model before subsequent processing steps that obscure measurement errors. Specifically, we model depth formation as a combined process of signal-dependent noise addition and non-uniform log-based quantization. The designed model is validated (with parameters fitted) using collected empirical data from a representative depth sensor. To enhance each pixel row in a depth image, we first encode intra-view similarities between available row pixels as edge weights via feature graph learning. We next establish inter-view similarities with another rectified depth image via viewpoint mapping and sparse linear interpolation. This leads to a maximum a posteriori (MAP) graph filtering objective that is convex and differentiable. We minimize the objective efficiently using accelerated gradient descent (AGD), where the optimal step size is approximated via Gershgorin circle theorem (GCT). Experiments show that our method significantly outperformed recent point cloud denoising schemes and state-of-the-art image denoising schemes in two established point cloud quality metrics.

Reconstructing the permittivity of dispersive scatterers from the measurements of scattered electromagnetic fields is a challenging problem due to the nonlinearity of the associated optimization problem. Traditionally, this has been addressed by collecting scattered field data at multiple frequencies and using lower frequency reconstructions as a priori information for higher frequency reconstructions. By modeling the object dispersion as a Debye medium, we propose an inversion technique that recovers the object permittivity with a minimum number of frequencies. We compare the performance of this method with our recently developed deep learning based technique (Sanghvi. et al., IEEE Trans. Comp. Imag., 2019) and show that given a properly trained neural network, single frequency reconstructions can be very competitive with multifrequency techniques that do not use neural networks. We quantify this performance via extensive numerical examples and comment on the hardware implications of both approaches.