A conventional camera captures an image of a directly visible scene by measuring the light intensity (and color) arriving at each pixel of its image sensor from a corresponding scene patch. Accordingly, conventional photography treats the measured light as being solely informative of the directly visible scene. Recent research has shown that subtle variations in measured light intensity can carry information about scenes outside the camera’s line of sight. A subset of these methods exploits preexisting obstructions—i.e., occluders— which cast barely perceptible yet highly informative soft shadows onto the observed planar surface. Whereas most prior works assume that exploitable occluders are either partly or wholly known or almost planar, a recent work blended a trained diffusion-based sampler to reconstruct the hidden occluding structures in 3D with a transverse 2D radiosity map of all other hidden non-occluding structures. This work proposes a fully trained novel multipath decoding UNet (MDUNet) architecture, in which the multimodal, multipath decoder parallels recent physics-based methods that achieve success by explicitly separating the representations and reconstructions of occluding and non-occluding hidden scene structures. However, by sharing a latent feature representation between the occluding and non-occluding structures, MDUNet couples their reconstruction pathways. Empirical results show that MDUNet improves inference times by over 100× compared to a state-of-the-art diffusion-based method and by 1000× compared to an iterative optimization-based method, while also improving reconstruction quality. In addition, MDUNet is trained solely on simulation data but generalizes to real experimental data, maintaining accuracy and stability even as ambient illumination increases. Code and training data are available at https://github.com/Predstan/MDUNet.

We demonstrate high-quality 3D non-line-of-sight imaging by exploiting a two-edge occluder and a scene representation constructed through a Fisher information analysis. Our reconstructions are computed from a single 2D photograph, and like many computational imaging problems, this inverse problem is ill-conditioned. The conditioning depends, in part, on the discretization of the hidden scene, both in terms of the coarseness and shape of the discretization grid. We show that a non-uniform discretization constructed to equalize the Fisher information metric improves the conditioning of the inverse problem for a desired reconstruction resolution. Experimental results demonstrate an improvement in condition number of over three orders, enabling more detailed and visually accurate reconstructions without increasing the reconstruction resolution.

The ability to image a scene without a direct line of sight has been demonstrated by exploiting measurements of light transients, speckle correlations, or even soft shadows (i.e., penumbra). Here, we present a new formulation of computational periscopy with an ordinary camera: a recent method that reconstructs two-dimensional images of a hidden scene from a single photograph of penumbra by exploiting the presence of a partially known occluder in the hidden scene. Our reformulation facilitates the recovery of three-dimensional information of the non-line-of-sight (NLOS) scene from an ordinary photograph of the penumbra produced on a visible matte surface. Specifically, we decompose the hidden scene into light-reflecting components constrained to be on a transverse 2D plane, and light-occluding components represented as a coarse 3D binary-valued occupancy grid. This decomposition, thus, yields a separable non-linear least squares inverse problem for reconstructing the hidden scene, using alternating minimization methods.

In a built environment, wanting to see without direct line of sight is often due to being outside a doorway. The two vertical edges of the doorway provide occlusions that can be exploited for non-line-of-sight imaging by forming corner cameras. While each corner camera can separately yield a robust 1D reconstruction, joint processing suggests novelties in both forward modeling and inversion. The resulting doorway camera provides accurate and robust 2D reconstructions of the hidden scene. This work provides a novel inversion algorithm to jointly estimate two views of change in the hidden scene, using the temporal difference between photographs acquired on the visible side of the doorway. Successful reconstruction is demonstrated in a variety of real and rendered scenarios, including different hidden scenes and lighting conditions. A Cramer-Rao bound analysis is used to demonstrate the 2D resolving power of the doorway camera over other passive acquisition strategies and to motivate the novel biangular reconstruction grid.

Edge-resolved transient imaging (ERTI) is a method for non-line-of-sight imaging that combines the use of direct time of flight for measuring distances with the azimuthal angular resolution afforded by a vertical edge occluder. Recently conceived and demonstrated for the first time, no performance analyses or optimizations of ERTI have appeared in published papers. This paper explains how the difficulty of detection of hidden scene objects with ERTI depends on a variety of parameters, including illumination power, acquisition time, ambient light, visible-side reflectivity, hidden-side reflectivity, target range, and target azimuthal angular position. Based on this analysis, optimization of the acquisition process is introduced whereby the illumination dwell times are varied to counteract decreasing signal-to-noise ratio at deeper angles into the hidden volume. Inaccuracy caused by a coaxial approximation is also analyzed and simulated.

We demonstrate non-line-of-sight imaging of multi-depth scenes using only a single photograph from an ordinary digital camera. The hidden scene, comprising two images at different depths, is partially occluded from a visible wall by an opaque occluding object. The distance from the visible wall to the hidden surfaces, and the images they contain, are recovered.

The ability to image scenery outside a camera's line-of-sight would be useful in a variety of applications, including autonomous vehicle collision avoidance, or for first responders to anticipate danger around a corner. When a wall obstructs the camera, light cast onto the floor from behind the wall may be used to recover angular variation of light intensity reflected by the hidden scene, forming a 1D scene projection. Recent work has demonstrated that temporal variation in a video, or sequence of floor images, may be used to image moving components of the hidden scene. However, in many applications, it would be useful to be able to image stationary components as well. This earlier approach was also designed for, and tested on, floors that have approximately uniform albedo, while many real floors have spatially varying albedo patterns such as checkered tiles and patterned carpets. In this work, we propose a method to reconstruct a 1D projection of all components in a hidden scene from a single photograph of the floor without assuming uniform floor albedo. Specifically, we derive a forward model that describes the measured photograph as a nonlinear combination of the unknown floor albedo and the light from behind the wall. The inverse problem, which is the joint estimation of floor albedo and a 1D reconstruction of the hidden scene, is then solved via optimization, where we introduce regularizers that help separate light variations in the measured photograph due to floor pattern and hidden scene, respectively. We demonstrate the effectiveness of our formulation and algorithm using synthetic and experimentally measured data.

We address the problem of estimating the parameter of a Bernoulli process. This arises in many applications, including photon-efficient active imaging where each illumination period is regarded as a single Bernoulli trial. We introduce a framework within which to minimize the mean-squared error (MSE) subject to an upper bound on the mean number of trials. This optimization has several simple and intuitive properties when the Bernoulli parameter has a beta prior. In addition, by exploiting typical spatial correlation using total variation regularization, we extend the developed framework to a rectangular array of Bernoulli processes representing the pixels in a natural scene. In simulations inspired by realistic active imaging scenarios, we demonstrate a 4.26 dB reduction in MSE due to the adaptive acquisition, as an average over many independent experiments and invariant to a factor of 3.4 variation in trial budget.

We present a framework for estimating non-localized sources of diffusion fields using spatiotemporal measurements of the field. Specifically in this contribution, we consider two non-localized source types: straight line and polygonal sources and assume that the induced field is monitored using a sensor network. Given the sensor measurements, we demonstrate, for each non-point source parameterization, how to reduce the source estimation problem to a system governed by a power series expansion that can then be efficiently solved using Prony's method, in order to reconstruct the source. We then evaluate the proposed algorithms by performing some numerical simulations using both noiseless and noisy spatiotemporal sensor measurements of the field.

In this contribution, we implement a fully distributed diffusion field estimation algorithm based on the use of average consensus schemes. We show that the field reconstruction problem is equivalent to estimating the sources of the field, and then derive an exact inversion formula for jointly recovering these sources when they are localized and instantaneous. Next we adapt this formula to the sensor network setting when only spatiotemporal samples of the field are available, and only local interactions between the sensors are allowed. To this end, we propose a robust distributed algorithm for reconstructing two-dimensional diffusion fields, sampled with a network of arbitrarily placed sensors. The proposed distributed algorithm is validated through numerical simulations in the noisy, multiple source setting.

In this paper we consider a diffusion field induced by multiple point sources and address the problem of reconstructing the field from its spatio-temporal samples obtained using a sensor network. We begin by formulating the problem as a multi-source estimation problem - so estimating source locations, activation times and intensities given samples of the induced field. Next a two-step algorithm is proposed for the single (localized and instantaneous) source field. First, the source location and intensity are estimated by applying the “reciprocity gap” principle; we show that this step can also reveal locations of multiple non-instantaneous sources. In the second step, we use an iterative method, based on Cauchy-Schwarz inequality, to find the activation time given the estimated location and intensity. Finally we extend this algorithm to the multi-source field and present simulation results to validate our findings.