Depth estimation and 3D object detection are critical for scene understanding but remain challenging to perform with a single image due to the loss of 3D information during image capture. Recent models using deep neural networks have improved monocular depth estimation performance, but there is still difficulty in predicting absolute depth and generalizing outside a standard dataset. Here we introduce the paradigm of deep optics, i.e. end-to-end design of optics and image processing, to the monocular depth estimation problem, using coded defocus blur as an additional depth cue to be decoded by a neural network. We evaluate several optical coding strategies along with an end-to-end optimization scheme for depth estimation on three datasets, including NYU Depth v2 and KITTI. We find an optimized freeform lens design yields the best results, but chromatic aberration from a singlet lens offers significantly improved performance as well. We build a physical prototype and validate that chromatic aberrations improve depth estimation on real-world results. In addition, we train object detection networks on the KITTI dataset and show that the lens optimized for depth estimation also results in improved 3D object detection performance.

Light field microscopy (LFM) is an emerging technique for volumetric fluorescence imaging, but widespread use is hampered by its poor spatial resolution. Using diffraction-based analysis we show how this degraded resolution arises because conventional LFM aims to sample four dimensions of the light field. By instead prioritizing 3D volumetric information over 4D sampling, we can optically interfere certain redundant angular samples to allow higher spatial resolution while maintaining enough angular information for depth discrimination. With this in mind, we design a number of aperture plane sampling schemes, characterize their frequency support and invertibility, and describe how their relative performance depends on the operating signal-to-noise regime. With simulations and a prototype, we demonstrate a time-sequential amplitude mask-based acquisition approach that outperforms conventional LFM in terms of both spatial resolution and axial field of view.

Light fields have many applications in machine vision, consumer photography, robotics, and microscopy. However, the prevalent resolution limits of existing light field imaging systems hinder widespread adoption. In this paper, we analyze fundamental resolution limits of light field cameras in the diffraction limit. We propose a sequential, coded-aperture-style acquisition scheme that optimizes the resolution of a light field reconstructed from multiple photographs captured from different perspectives and fnumber settings. We also show that the proposed acquisition scheme facilitates high dynamic range light field imaging and demonstrate a proof-of-concept prototype system. With this work, we hope to advance our understanding of the resolution limits of light field photography and develop practical computational imaging systems to overcome them.