Image enhancement is a long-studied problem that is understood to be fundamentally ill-posed, meaning that there are many high-quality images that are consistent with any given low-quality observation. When image enhancement is applied to biometric samples, such as sharpening or super-resolving a face image, this ill-posedness is often ignored and a single high-quality sample is estimated without ensuring that it's consistent with the observation. In this work, we describe a method to enumerate multiple high-quality samples from a single input, all of which are consistent with the low-quality input, and use this to estimate confidence in a face-based search result. This method quantifies the sample- and gallery-conditioned uncertainty by enumerating multiple high-quality images from a single low-quality sample, ensuring that each is consistent with the input. We demonstrate this by first showing that, even with modern deep-face features, low-resolution face recognition is still ill-posed even when applied to frontal face images. We enumerate multiple high-resolution (HR) images that are consistent with a low-resolution (LR) face sample, represent these with modern deep features, and demonstrate a subject-specific ill-posedness in recognition.

Camera Response Functions (CRFs) map the irradiance incident at a sensor pixel to an intensity value in the corresponding image pixel. The nonlinearity of CRFs impact physics-based and low-level computer vision methods like de-blurring, photometric stereo, etc. In addition, CRFs have been used for forensics to identify regions of an image spliced in from a different camera. Despite its importance, the process of radiometrically calibrating a camera's CRF is significantly harder and less standardized than geometric calibration. Competing methods use different mathematical models of the CRF, some of which are derived from an outdated dataset. We present a new dataset of 178 CRFs from modern digital cameras, derived from 1565 camera review images available online, and use it to answer a series of questions about CRFs. Which mathematical models are best for CRF estimation? How have they changed over time? And how unique are CRFs from camera to camera?

With the rise of misinformation spread via social media channels, enabled by the increasing automation and realism of image manipulation tools, image forensics is an increasingly relevant problem. Classic image forensic methods leverage low-level cues such as metadata, sensor noise fingerprints, and others that are easily fooled when the image is re-encoded upon upload to facebook, etc. This necessitates the use of higher-level physical and semantic cues that, once hard to estimate reliably in the wild, have become more effective due to the increasing power of computer vision. In particular, we detect manipulations introduced by artificial blurring of the image, which creates inconsistent photometric relationships between image intensity and various cues. We achieve 98% accuracy on the most challenging cases in a new dataset of blur manipulations, where the blur is geometrically correct and consistent with the scene's physical arrangement. Such manipulations are now easily generated, for instance, by smartphone cameras having hardware to measure depth, e.g. 'Portrait Mode' of the iPhone7Plus. We also demonstrate good performance on a challenge dataset evaluating a wider range of manipulations in imagery representing 'in the wild' conditions.

Recent advances on image manipulation techniques have made image forgery detection increasingly more challenging. An important component in such tools is to fake motion and/or defocus blurs through boundary splicing and copy-move operators, to emulate wide aperture and slow shutter effects. In this paper, we present a new technique based on the analysis of the camera response functions (CRF) for efficient and robust splicing and copy-move forgery detection and localization. We first analyze how non-linear CRFs affect edges in terms of the intensity-gradient bivariable histograms. We show distinguishable shape differences on real vs. forged blurs near edges after a splicing operation. Based on our analysis, we introduce a deep-learning framework to detect and localize forged edges. In particular, we show the problem can be transformed to a handwriting recognition problem an resolved by using a convolutional neural network. We generate a large dataset of forged images produced by splicing followed by retouching and comprehensive experiments show our proposed method outperforms the state-of-the-art techniques in accuracy and robustness.

We describe a technique to recover depth from a light field (LF) using two proposed features of the LF focal stack. One feature is the property that non-occluding pixels exhibit symmetry along the focal depth dimension centered at the in-focus slice. The other is a data consistency measure based on analysis-by-synthesis, i.e., the difference between the synthesized focal stack given the hypothesized depth map and that from the LF. These terms are used in an iterative optimization framework to extract scene depth. Experimental results on real Lytro and Raytrix data demonstrate that our technique outperforms state-of-the-art solutions and is significantly more robust to noise and under-sampling.

In this paper, we introduce a bilateral consistency metric on the surface camera (SCam) [26] for light field stereo matching to handle significant occlusions. The concept of SCam is used to model angular radiance distribution with respect to a 3D point. Our bilateral consistency metric is used to indicate the probability of occlusions by analyzing the SCams. We further show how to distinguish between on-surface and free space, textured and non-textured, and Lambertian and specular through bilateral SCam analysis. To speed up the matching process, we apply the edge preserving guided filter [14] on the consistency-disparity curves. Experimental results show that our technique outperforms both the state-of-the-art and the recent light field stereo matching methods, especially near occlusion boundaries.