Computational approach to imaging around the corner, or non-line-of-sight (NLOS) imaging, is becoming a reality thanks to major advances in imaging hardware and reconstruction algorithms. A recent development towards practical NLOS imaging, (Nam et al. 2021) demonstrated a high-speed non-confocal imaging system that operates at 5Hz, 100x faster than the prior art. This enormous gain in acquisition rate, however, necessitates numerous approximations in light transport, breaking many existing NLOS reconstruction methods that assume an idealized image formation model. To bridge the gap, we present a novel deep model that incorporates the complementary physics priors of wave propagation and volume rendering into a neural network for high-quality and robust NLOS reconstruction. This orchestrated design regularizes the solution space by relaxing the image formation model, resulting in a deep model that generalizes well on real captures despite being exclusively trained on synthetic data. Further, we devise a unified learning framework that enables our model to be flexibly trained using diverse supervision signals, including target intensity images or even raw NLOS transient measurements. Once trained, our model renders both intensity and depth images at inference time in a single forward pass, capable of processing more than 5 captures per second on a high-end GPU. Through extensive qualitative and quantitative experiments, we show that our method outperforms prior physics and learning based approaches on both synthetic and real measurements. We anticipate that our method along with the fast capturing system will accelerate future development of NLOS imaging for real world applications that require high-speed imaging.

In this article, the cluster consensus of heterogeneous multiagent systems is investigated via dynamic event-triggered and pining control strategies. Comprehensively considering the cooperative and competitive interactions among the agents, a novel cluster consensus control protocol is designed, and a class of fully distributed dynamic event-triggered laws that do not depend on the global information is proposed. Being distinct from the traditional static threshold value, a time-varying threshold value is introduced which can effectively reduce the trigger time. Meanwhile, some sufficient conditions are obtained theoretically by using Lyapunov stability theory and the pinning control strategy is addressed as well. Furthermore, it has also been shown that each agent can exclude Zeno behavior. Finally, two simulated examples are given to illustrate the correctness and validity of the proposed methods and our findings.

Non-Line-of-Sight (NLOS) imaging reconstructs occluded scenes based on indirect diffuse reflections. The computational complexity and memory consumption of existing NLOS reconstruction algorithms make them challenging to be implemented in real-time. This paper presents a fast and memory-efficient phasor field-diffraction-based NLOS reconstruction algorithm. In the proposed algorithm, the radial property of the Rayleigh Sommerfeld diffraction (RSD) kernels along with the linear property of Fourier transform are utilized to reconstruct the Fourier domain representations of RSD kernels using a set of kernel bases. Moreover, memory consumption is further reduced by sampling the kernel bases in a radius direction and constructing them during the run-time. According to the analysis, the memory efficiency can be improved by as much as 220×. Experimental results show that compared with the original RSD algorithm, the reconstruction time of the proposed algorithm is significantly reduced with little impact on the final imaging quality.

Non-line-of-sight (NLOS) imaging systems reconstruct hidden scenes using computational methods based on indirect light that diffusely reflected from relay walls. Due to the computation and memory requirements of reconstruction algorithms, real-time NLOS imaging for room-size scenes based on non-confocal data has long been challenging. This paper proposes a field programmable gate array (FPGA) accelerator for the recently proposed Rayleigh-Sommerfeld Diffraction (RSD)-based NLOS reconstruction method. In the proposed accelerator design, ring sampling and radius sampling techniques are proposed to reduce the memory requirements by reconstructing the RSD kernels with a set of kernel bases and ring sampling coefficients during the runtime. Based on that, a customized hardware architecture and the corresponding FPGA design for real-time RSD-based NLOS reconstruction is further proposed. Implementation results show that the proposed FPGA accelerator is capable of reconstructing NLOS scenes at 25 frames per second (FPS), running at a relatively slow clock frequency of 50 MHz. To the best knowledge of the authors, this is the first real-time enabled FPGA accelerator for room-size NLOS imaging with a resolution of$128\times 128$.

The light transport matrix (LTM) is an instrumental tool in line-of-sight (LOS) imaging, describing how light interacts with the scene and enabling applications such as relighting or separation of illumination components. We introduce a framework to estimate the LTM of non-line-of-sight (NLOS) scenarios, coupling recent virtual forward light propagation models for NLOS imaging with the LOS light transport equation. We design computational projector-camera setups, and use these virtual imaging systems to estimate the transport matrix of hidden scenes. We introduce the specific illumination functions to compute the different elements of the matrix, overcoming the challenging wide-aperture conditions of NLOS setups. Our NLOS light transport matrix allows us to (re)illuminate specific locations of a hidden scene, and separate direct, first-order indirect, and higher-order indirect illumination of complex cluttered hidden scenes, similar to existing LOS techniques.

Non-line-of-sight (NLOS) imaging aims to visualize occluded scenes by exploiting indirect reflections on visible surfaces. Previous methods approach this problem by inverting the light transport on the hidden scene, but are limited to isolated, diffuse objects. The recently introduced phasor fields framework computationally poses NLOS reconstruction as a virtual line-of-sight (LOS) problem, lifting most assumptions about the hidden scene. In this work we complement recent theoretical analysis of phasor field-based reconstruction, by empirically analyzing the effect of reflectance of the hidden scenes on reconstruction. We experimentally study the reconstruction of hidden scenes composed of objects with increasingly specular materials. Then, we evaluate the effect of the virtual aperture size on the reconstruction, and establish connections between the effect of these two different dimensions on the results. We hope our analysis helps to characterize the imaging capabilities of this promising new framework, and foster new NLOS imaging modalities.

Non-Line-of-Sight imaging has been linked to wave diffraction by the recent phasor field method. In wave optics, the Wigner Distribution Function description for an optical imaging system is a powerful analytical tool for modeling the imaging process with geometrical transformations. In this paper, we focus on illustrating the relation between captured signals and hidden objects in the Wigner Distribution domain. The Wigner Distribution Function is usually used together with approximated diffraction propagators, which is fine for most imaging problems. However, these approximated diffraction propagators are not valid for Non-Line-of-Sight imaging scenarios. We show that the exact phasor field propagator (Rayleigh-Sommerfeld Diffraction) does not have a standard geometrical transformation, as compared to approximated diffraction propagators (Fresnel, Fraunhofer diffraction) that can be represented as shearing or rotation in the Wigner Distribution Function domain. Then, we explore differences between the exact and approximated solutions by characterizing errors made in different spatial positions and acquisition methods (confocal, non-confocal scanning). We derive a lateral resolution based on the exact phasor field propagator, which can be used as a reference for theoretical evaluations and comparisons. For targets that lie laterally outside a relay wall, the loss of resolution is geometrically illustrated in the context of the Wigner Distribution Function.

We formulate an equation describing a general Non-line-of-sight (NLOS) imaging measurement and analyze the properties of the measurement in the Fourier domain regarding the spatial frequencies of the scene it encodes. We conclude that for a relay wall with finite size, certain scene configurations and features are not detectable in an NLOS measurement. We then provide experimental examples of invisible scene features and their reconstructions, as well as a set of example scenes that lead to an ill-posed NLOS imaging problem.

We present a novel compositional method for background subtraction. The presented method can better handle multi-modal appearance of the background in a dynamic environment. In this method, an illumination compensation method based on Local Binary Pattern is used to compensate sudden global and/or local illumination changes, and it is also used for filtering the pixels from background rather than foreground for background subtraction algorithms. Using this scheme, we can improve the accuracy of background model and make our method better handle the high-frequency illumination changes. An efficient compositional scheme is proposed to combine the parameter-based and the samples-based algorithm for background subtraction. Thanks to this scheme, we can get more reliable results than any of them. A new initialization technique for Gaussian Mixture Model based on spatial information is also presented, which can help us get meaningful results from the second frame. Experimental results clearly justify our method.