What is the current state-of-the-art for image restoration and enhancement applied to degraded images acquired under less than ideal circumstances? Can the application of such algorithms as a pre-processing step improve image interpretability for manual analysis or automatic visual recognition to classify scene content? While there have been important advances in the area of computational photography to restore or enhance the visual quality of an image, the capabilities of such techniques have not always translated in a useful way to visual recognition tasks. Consequently, there is a pressing need for the development of algorithms that are designed for the joint problem of improving visual appearance and recognition, which will be an enabling factor for the deployment of visual recognition tools in many real-world scenarios. To address this, we introduce the UG$^2$dataset as a large-scale benchmark composed of video imagery captured under challenging conditions, and two enhancement tasks designed to test algorithmic impact on visual quality and automatic object recognition. Furthermore, we propose a set of metrics to evaluate the joint improvement of such tasks as well as individual algorithmic advances, including a novel psychophysics-based evaluation regime for human assessment and a realistic set of quantitative measures for object recognition performance. We introduce six new algorithms for image restoration or enhancement, which were created as part of the IARPA sponsored UG$^2$Challenge workshop held at CVPR 2018. Under the proposed evaluation regime, we present an in-depth analysis of these algorithms and a host of deep learning-based and classic baseline approaches. From the observed results, it is evident that we are in the early days of building a bridge between computational photography and visual recognition, leaving many opportunities for innovation in this area.

A transmission matrix describes the input-output relationship of a complex wavefront as it passes through/reflects off a multiple-scattering medium, such as frosted glass or a painted wall. Knowing a medium's transmission matrix enables one to image through the medium, send signals through the medium, or even use the medium as a lens. The double phase retrieval method is a recently proposed technique to learn a medium's transmission matrix that avoids difficult-to-capture interferometric measurements. Unfortunately, to perform high resolution imaging, existing double phase retrieval methods require (1) a large number of measurements and (2) an unreasonable amount of computation. In this work we focus on the latter of these two problems and reduce computation times with two distinct methods: First, we develop a new phase retrieval algorithm that is significantly faster than existing methods, especially when used with an amplitude-only spatial light modulator (SLM). Second, we calibrate the system using a phase-only SLM, rather than an amplitude-only SLM which was used in previous double phase retrieval experiments. This seemingly trivial change enables us to use a far faster class of phase retrieval algorithms. As a result of these advances, we achieve a 100x reduction in computation times, thereby allowing us to image through scattering media at state-of-the-art resolutions. In addition to these advances, we also release the first publicly available transmission matrix dataset. This contribution will enable phase retrieval researchers to apply their algorithms to real data. Of particular interest to this community, our measurement vectors are naturally i.i.d. subgaussian, i.e., no coded diffraction pattern is required.

We consider the problem of modelling asymmetric pulse trains as finite-rate-of-innovation (FRI) signals. In particular, we show that the sum of amplitude-scaled and time-shifted pulses with different asymmetry factors is an FRI signal. Such signals frequently arise in applications such as ultrasound and radio detection and ranging (RADAR) where the received signal has skewed pulses. In this paper, we model the asymmetric component of a pulse using its derivative. A sampling kernel with a sum-of-sincs frequency response is used to measure the samples, and a modified annihilating filter method is applied on the samples to estimate the parameters of the FRI signal. We show accurate reconstruction for signals containing asymmetric Gaussian, Cauchy-Lorentz, and sinc pulses. Analysis of the proposed scheme in the presence of noise shows that the error in the estimated parameters decreases by oversampling the signal.

We consider the problem of parameter estimation from real-valued multi-tone signals. Such problems arise frequently in spectral estimation. More recently, they have gained new importance in finite-rate-of-innovation signal sampling and reconstruction. The annihilating filter is a key tool for parameter estimation in these problems. The standard annihilating filter design has to be modified to result in accurate estimation when dealing with real sinusoids, particularly because the real-valued nature of the sinusoids must be factored into the annihilating filter design. We show that the constraint on the annihilating filter can be relaxed by making use of the Hilbert transform. We refer to this approach as the Hilbert annihilating filter approach. We show that accurate parameter estimation is possible by this approach. In the single-tone case, the mean-square error performance increases by 6 dB for signal-to-noise ratio (SNR) greater than 0 dB. We also present experimental results in the multi-tone case, which show that a significant improvement (about 6dB) is obtained when the parameters are close to 0 or π. In the mid-frequency range, the improvement is about 2 to 3dB.

Recently, a method of finding the spectral samples of non-periodic-finite-rate-of-innovation (NP-FRI) signals using a sum-of-sincs (SoS) sampling kernel was proposed in the literature. In the SoS approach, the kernel is repeated at a rate dependent on the delays of the FRI signal. The number of repetitions depends on both the duration and the delays of pulses constituting the FRI signal. In this paper, we show that the kernel repetition can be avoided and perfect reconstruction can be obtained by working with the SoS kernel directly provided that certain sampling criteria are satisfied. We place a lower bound on the sampling rate to ensure that exact signal reconstruction is achieved using filtered samples. To suppress the effect of noise, we use Cadzow denoising technique. Reconstruction is achieved using the annihilating filter method. We report results on data simulated using Field II software as well as real cardiac ultrasound data. The experimental results show that, with nearly 10 times less data than that required by the standard technique, the proposed method gives a comparable quality of reconstruction. The reconstruction accuracy can be controlled by choosing the model order of the NP-FRI signal appropriately.