Traditional hand-held light field cameras only observe a small fraction of the cone of light emitted by a scene point. As a consequence, the study of interesting angular effects like iridescence are beyond the scope of such cameras. This paper envisions a new design for sensing light fields with wide baselines, so as to sense a significantly larger fraction of the cone of light emitted by scene points. Our system achieves this by imaging the scene, indirectly, through an ellipsoidal mirror. We show that an ellipsoidal mirror maps a wide cone of light from locations near one of its foci to a narrower cone at its other focus; thus, by placing a conventional light field camera at a focus, we can observe a wide-baseline light field from the scene near the other focus. We show via simulations and a lab prototype that wide-baseline light fields excel in the traditional applications involving changes in focus and perspective. Additionally, the larger cone of light that they observe allows the study of iridescence and thin-film interference. Perhaps surprisingly, the larger cone of light allows us to estimate surface normals of scene points by reasoning about their visibility.

Acquisition of high-dimensional spatially-varying bidirectional reflectance distribution functions (SVBRDFs) is extremely challenging for scenes that exhibit high-frequency angular effects-in particular, iridescence-that requires dense sampling across spatial and angular dimensions of reflectance. This is further complicated by the need to illuminate and observe the material along grazing angles, where the effects of Iridescence are often most prominent. This paper proposes an imaging system for acquiring the reflectance associated with such iridescent materials. Our system uses an imaging setup, consisting of an ellipsoidal mirror and a light field camera, that can acquire a dense slice of the SVBRDF across spatial and observation angle axes from a single image, including at grazing observation angles. We use an active illumination setup to sparsely sample the incident illumination angles, thereby providing an acquisition setup that only requires a few (light field) photographs. To compensate for the lack of density in the incident angles, we train a neural model that implicitly represents the SVBRDF, creating a light-weight data-driven reflectance function that enables interpolation of the missing measurements. We show that our imaging system can reconstruct textures that exhibit spatially-varying iridescence stemming from diffraction as well as structural coloration.

We introduce a method that recovers full-surround 3D reconstructions from a single kaleidoscopic image using a neural surface representation. Full-surround 3D reconstruction is critical for many applications, such as augmented and virtual reality. A kaleidoscope, which uses a single camera and multiple mirrors, is a convenient way of achieving full-surround coverage, as it redistributes light directions and thus captures multiple viewpoints in a single image. This enables single-shot and dynamic full-surround 3D reconstruction. However, using a kaleidoscopic image for multiview stereo is challenging, as we need to decompose the image into multi-view images by identifying which pixel corresponds to which virtual camera, a process we call labeling. To address this challenge, pur approach avoids the need to explicitly estimate labels, but instead “sculpts” a neural surface representation through the careful use of silhouette, background, foreground, and texture information present in the kaleidoscopic image. We demonstrate the advantages of our method in a range of simulated and real experiments, on both static and dynamic scenes.

We introduce a novel video-rate hyperspectral imager with high spatial, temporal and spectral resolutions. Our key hypothesis is that spectral profiles of pixels within each super-pixel tend to be similar. Hence, a scene-adaptive spatial sampling of a hyperspectral scene, guided by its super-pixel segmented image, is capable of obtaining high-quality reconstructions. To achieve this, we acquire an RGB image of the scene, compute its super-pixels, from which we generate a spatial mask of locations where we measure high-resolution spectrum. The hyperspectral image is subsequently estimated by fusing the RGB image and the spectral measurements using a learnable guided filtering approach. Due to low computational complexity of the superpixel estimation step, our setup can capture hyperspectral images of the scenes with little overhead over traditional snapshot hyperspectral cameras, but with significantly higher spatial and spectral resolutions. We validate the proposed technique with extensive simulations as well as a lab prototype that measures hyperspectral video at a spatial resolution of 600 ×900 pixels, at a spectral resolution of 10 nm over visible wavebands, and achieving a frame rate at 18fps.