Imaging systems have traditionally been designed to mimic the human eye and produce visually interpretable measurements. Modern imaging systems, however, process raw measurements computationally before or instead of human viewing. As a result, the information content of raw measurements matters more than their visual interpretability. Despite the importance of measurement information content, current approaches for evaluating imaging system performance do not quantify it: they instead either use alternative metrics that assess specific aspects of measurement quality or assess measurements indirectly with performance on secondary tasks. We developed the theoretical foundations and a practical method to directly quantify mutual information between noisy measurements and unknown objects. By fitting probabilistic models to measurements and their noise characteristics, our method estimates information by upper bounding its true value. By applying gradient-based optimization to these estimates, we also developed a technique for designing imaging systems called Information-Driven Encoder Analysis Learning (IDEAL). Our information estimates accurately captured system performance differences across four imaging domains (color photography, radio astronomy, lensless imaging, and microscopy). Systems designed with IDEAL matched the performance of those designed with end-to-end optimization, the prevailing approach that jointly optimizes hardware and image processing algorithms. These results establish mutual information as a universal performance metric for imaging systems that enables both computationally efficient design optimization and evaluation in real-world conditions. A video summary of this work can be found at: https://waller-lab.github.io/EncodingInformationWebsite/

The Transport of Intensity Equation (TIE) in microscopy and the Depth from Differential Defocus (DfDD) method in photography both describe the effect of a small change in defocus on image intensity. They are based on different assumptions and may appear to contradict each other. Using the Wigner Distribution Function, we show that DfDD can be interpreted as a special case of the TIE, well-suited to applications where the generalized phase measurements recovered by the TIE are connected to depth rather than phase, such as photography and fluorescence microscopy. The level of spatial coherence is identified as the driving factor in the trade-off between the usefulness of each technique. Specifically, the generalized phase corresponds to the sample's phase under high-coherence illumination and reveals scene depth in low-coherence settings. When coherence varies spatially, as in multi-modal phase and fluorescence microscopy, we show that complementary information is available in different regions of the image.