Wavefront shaping systems attempt to correct aberrations caused by tissue scattering, by placing a spatial light modulator (SLM) in the optical path and using it to reshape the light wave emitted from a target of interest deep inside the tissue. However, the field-of-view we can correct with one SLM pattern is extremely small, because inherently the aberration is caused by the 3D tissue structure while the SLM is planar. To overcome this, multi-conjugate correction systems built with multiple layered SLMs have been introduced, which attempt to approximate the 3D tissue structure with multiple planar aberrations.However, multi-conjugate systems with a large number of SLMs are not easy to construct. Therefore, it is important to understand how many SLM layers are actually needed for a good correction, and whether there are practical benefits from correction systems with a relatively small number of layers. To help in the design of future systems, this paper analyzes multi-layer corrections. We show that the well-known missing cone problem which fundamentally limits the axial resolution of microscopes, turns into an advantage when designing 3D correction systems. Due to this property less information should be captured by the correction system, facilitating sparser approximations. We also show that even if the number of available layers is insufficient for a reasonable approximation of the full 3D tissue volume, it can significantly expand the field-of-view when compared to single-layer correction systems, and therefore can largely accelerate their operation.

In material acquisition we want to infer the internal properties of materials from the way they scatter light. In particular, we are interested in measuring the phase function of the material, governing the amount of energy scattered towards different directions. This phase function has been shown to carry a lot of information about the type and size of particles dispersed in the medium, and is therefore essential for its characterization.

Coherent images of scattering materials, such as biological tissue, typically exhibit high-frequency intensity fluctuations known as speckle. These seemingly noise-like speckle patterns have strong statistical correlation properties that have been successfully utilized by computational imaging systems in different application areas. Unfortunately, these properties are not well-understood, in part due to the difficulty of simulating physically-accurate speckle patterns. In this work, we propose a new model for speckle statistics based on a single scattering approximation, that is, the assumption that all light contributing to speckle correlation has scattered only once. Even though single-scattering models have been used in computer vision and graphics to approximate intensity images due to scattering, such models usually hold only for very optically thin materials, where light indeed does not scatter more than once. In contrast, we show that the single-scattering model for speckle correlation remains accurate for much thicker materials. We evaluate the accuracy of the single-scattering correlation model through exhaustive comparisons against an exact speckle correlation simulator. We additionally demonstrate the model's accuracy through comparisons with real lab measurements. We show, that for many practical application settings, predictions from the single-scattering model are more accurate than those from other approximate models popular in optics, such as the diffusion and Fokker-Planck models. We show how to use the single-scattering model to derive closed-form expressions for speckle correlation, and how these expressions can facilitate the study of statistical speckle properties. In particular, we demonstrate that these expressions provide simple explanations for previously reported speckle properties, and lead to the discovery of new ones. Finally, we discuss potential applications for future computational imaging systems.

Recent advances in computational imaging have significantly expanded our ability to image through scattering layers such as biological tissues by exploiting the auto-correlation properties of captured speckle intensity patterns. However, most experimental demonstrations of this capability focus on the far-field imaging setting, where obscured light sources are very far from the scattering layer. By contrast, medical imaging applications such as fluorescent imaging operate in the near-field imaging setting, where sources are inside the scattering layer. We provide a theoretical and experimental study of the similarities and differences between the two settings, highlighting the increased challenges posed by the near-field setting. We then draw insights from this analysis to develop a new algorithm for imaging through scattering that is tailored to the near-field setting by taking advantage of unique properties of speckle patterns formed under this setting, such as their local support. We present a theoretical analysis of the advantages of our algorithm and perform real experiments in both far-field and near-field configurations, showing an order-of magnitude expansion in both the range and the density of the obscured patterns that can be recovered.

We present a Monte Carlo rendering framework for the physically-accurate simulation of speckle patterns arising from volumetric scattering of coherent waves. These noise-like patterns are characterized by strong statistical properties, such as the so-called memory effect. These properties are at the core of imaging techniques for applications as diverse as tissue imaging, motion tracking, and non-line-of-sight imaging. Our rendering framework can replicate these properties computationally, in a way that is orders of magnitude more efficient than alternatives based on directly solving the wave equations. At the core of our framework is a path-space formulation for the covariance of speckle patterns arising from a scattering volume, which we derive from first principles. We use this formulation to develop two Monte Carlo rendering algorithms, for computing speckle covariance as well as directly speckle fields. While approaches based on wave equation solvers require knowing the microscopic position of wavelength-sized scatterers, our approach takes as input only bulk parameters describing the statistical distribution of these scatterers inside a volume. We validate the accuracy of our framework by comparing against speckle patterns simulated using wave equation solvers, use it to simulate memory effect observations that were previously only possible through lab measurements, and demonstrate its applicability for computational imaging tasks.

Due to atmospheric turbulence, light randomly refracts in three dimensions (3D), eventually entering a camera at a perturbed angle. Each viewed object point thus has a distorted projection in a two-dimensional (2D) image. Simulating 3D random refraction for all viewed points via complex simulated 3D random turbulence is computationally expensive. We derive an efficient way to render 2D image distortions, consistent with turbulence. Our approach bypasses 3D numerical calculations altogether We directly create 2D random physics-based distortion vector fields, where correlations are derived in closed form from turbulence theory. The correlations are nontrivial: they depend on the perturbation directions relative to the orientation of all object-pairs, simultaneously. Hence, we develop a theory characterizing and rendering such a distortion field. The theory is turned to a few simple 2D operations, which render images based on camera and atmospheric properties.

Random refraction occurs in turbulence and through a wavy water-air interface. It creates distortion that changes in space, time and with viewpoint. Localizing objects in three dimensions (3D) despite this random distortion is important to some predators and also to submariners avoiding the salient use of periscopes. We take a multiview approach to this task. Refracted distortion statistics induce a probabilistic relation between any pixel location and a line of sight in space. Measurements of an object's random projection from multiple views and times lead to a likelihood function of the object's 3D location. The likelihood leads to estimates of the 3D location and its uncertainty. Furthermore, multiview images acquired simultaneously in a wide stereo baseline have uncorrelated distortions. This helps reduce the acquisition time needed for localization. The method is demonstrated in stereoscopic video sequences, both in a lab and a swimming pool.

Viewing an airborne scene from a submerged camera creates a virtual periscope, avoiding the saliency of a real maritime periscope. Random waves in the water-air interface severely distort the view, by refraction. We show a way to handle this. The distortion can be significantly countered, based on an estimate of the wavy water interface, at the instant of imaging. We obtain the interface estimate per frame, by a submerged sensor that simultaneously images the refracted Sun through an array of submerged pinholes. Our use of a stellar cue for estimation (and then correction) of the refractive field has analogy to ground-based astronomy. In astronomy, a guide star is imaged by a Shack-Hartmann sensor, for an estimate of the random refractive field created by atmospheric turbulence. In astronomy, this principle is used for countering blur, mainly by adaptive optics, while we use it for compensating distortions created by water waves. We introduce this novel concept for enabling a virtual periscope, demonstrate it, and analyze some of its limitations.

Random refraction occurs in turbulence and through a wavy water-air interface. It creates distortion that changes in space, time and with viewpoint. Localizing objects in three dimensions (3D) despite this random distortion is important to some predators and also to submariners avoiding the salient use of periscopes. We take a multiview approach to this task. Refracted distortion statistics induce a probabilistic relation between any pixel location and a line of sight in space. Measurements of an object's random projection from multiple views and times lead to a likelihood function of the object's 3D location. The likelihood leads to estimates of the 3D location and its uncertainty. Furthermore, multiview images acquired simultaneously in a wide stereo baseline have uncorrelated distortions. This helps reduce the acquisition time needed for localization. The method is demonstrated in stereoscopic video sequences, both in a lab and a swimming pool.

Refraction causes random dynamic distortions in atmospheric turbulence and in views across a water interface. The latter scenario is experienced by submerged animals seeking to detect prey or avoid predators, which may be airborne or on land. Man encounters this when surveying a scene by a submarine or divers while wishing to avoid the use of an attention-drawing periscope. The problem of inverting random refracted dynamic distortions is difficult, particularly when some of the objects in the field of view (FOV) are moving. On the other hand, in many cases, just those moving objects are of interest, as they reveal animal, human, or machine activity. Furthermore, detecting and tracking these objects does not necessitate handling the difficult task of complete recovery of the scene. We show that moving objects can be detected very simply, with low false-positive rates, even when the distortions are very strong and dominate the object motion. Moreover, the moving object can be detected even if it has zero mean motion. While the object and distortion motions are random and unknown, they are mutually independent. This is expressed by a simple motion feature which enables discrimination of moving object points versus the background.