Time-correlated single-photon technology is emerging as an important approach to 3D Imaging. This paper presents a reconstruction algorithm that exploits data statistics and multi-scale information to deliver clean depth and reflectivity images together with associated uncertainty maps. The statistical method has been implemented to run on graphics processing units (GPUs) that enable real-time reconstruction of moving scenes at more than 1000 depth frames per second on the 32 × 64 pixels real Quantic4x4 SPAD sensor array data. Comparisons with state-of-the-art algorithms on simulated and real data demonstrate the robust and efficient performance of the proposed method.

This paper presents a new Bayesian algorithm for the robust reconstruction of multispectral single-photon Lidar data acquired in extreme conditions. We focus on imaging through obscurants (i.e., fog, water) leading to high and possibly non-uniform background noise. The proposed hierarchical Bayesian method accounts for multiscale information to provide distribution estimates for the target’s depth and reflectivity, i.e., point and uncertainty measures of the estimates to improve decision making. The correlations between variables are enforced using a weighting scheme that allows the incorporation of guide information available from other sensors or state-of-the-art algorithms. Results on synthetic and real data show improved reconstruction of the scene in extreme conditions when compared to the state-of-the-art algorithms.

In this article, we present a new algorithm for fast, online 3D reconstruction of dynamic scenes using times of arrival of photons recorded by single-photon detector arrays. One of the main challenges in 3D imaging using single-photon lidar in practical applications is the presence of strong ambient illumination which corrupts the data and can jeopardize the detection of peaks/surface in the signals. This background noise not only complicates the observation model classically used for 3D reconstruction but also the estimation procedure which requires iterative methods. In this work, we consider a new similarity measure for robust depth estimation, which allows us to use a simple observation model and a non-iterative estimation procedure while being robust to mis-specification of the background illumination model. This choice leads to a computationally attractive depth estimation procedure without significant degradation of the reconstruction performance. This new depth estimation procedure is coupled with a spatio-temporal model to capture the natural correlation between neighboring pixels and successive frames for dynamic scene analysis. The resulting online inference process is scalable and well suited for parallel implementation. The benefits of the proposed method are demonstrated through a series of experiments conducted with simulated and real single-photon lidar videos, allowing the analysis of dynamic scenes at 325 m observed under extreme ambient illumination conditions.

This paper presents a new algorithm for the learning of spatial correlation and non-local restoration of single-photon 3-Dimensional Lidar images acquired in the photon starved regime (fewer or less than one photon per pixel) or with a reduced number of scanned spatial points (pixels). The algorithm alternates between three steps: (i) extract multi-scale information, (ii) build a robust graph of non-local spatial correlations between pixels, and (iii) the restoration of depth and reflectivity images. A non-uniform sampling approach, which assigns larger patches to homogeneous regions and smaller ones to heterogeneous regions, is adopted to reduce the computational cost associated with the graph. The restoration of the 3D images is achieved by minimizing a cost function accounting for the multi-scale information and the non-local spatial correlation between patches. This minimization problem is efficiently solved using the alternating direction method of multipliers (ADMM) that presents fast convergence properties. Various results based on simulated and real Lidar data show the benefits of the proposed algorithm that improves the quality of the estimated depth and reflectivity images, especially in the photon-starved regime or when containing a reduced number of spatial points.

Light detection and ranging (Lidar) data can be used to capture the depth and intensity profile of a 3D scene. This modality relies on constructing, for each pixel, a histogram of time delays between emitted light pulses and detected photon arrivals. In a general setting, more than one surface can be observed in a single pixel. The problem of estimating the number of surfaces, their reflectivity, and position becomes very challenging in the low-photon regime (which equates to short acquisition times) or relatively high background levels (i.e., strong ambient illumination). This paper presents a new approach to 3D reconstruction using single-photon, single-wavelength Lidar data, which is capable of identifying multiple surfaces in each pixel. Adopting a Bayesian approach, the 3D structure to be recovered is modelled as a marked point process, and reversible jump Markov chain Monte Carlo (RJ-MCMC) moves are proposed to sample the posterior distribution of interest. In order to promote spatial correlation between points belonging to the same surface, we propose a prior that combines an area interaction process and a Strauss process. New RJ-MCMC dilation and erosion updates are presented to achieve an efficient exploration of the configuration space. To further reduce the computational load, we adopt a multiresolution approach, processing the data from a coarse to the finest scale. The experiments performed with synthetic and real data show that the algorithm obtains better reconstructions than other recently published optimization algorithms for lower execution times.

This paper presents a novel nonlinear hyperspectral mixture model and its associated supervised unmixing algorithm. The model assumes a linear mixing model corrupted by an additive term which accounts for multiple scattering nonlinearities (NL). The proposed model generalizes bilinear models by taking into account higher order interaction terms. The inference of the abundances and nonlinearity coefficients of this model is formulated as a convex optimization problem suitable for fast estimation algorithms. This formulation accounts for constraints such as the sum-to-one and nonnegativity of the abundances, the non-negativity of the nonlinearity coefficients, and the spatial sparseness of the residuals. The resulting convex problem is solved using the alternating direction method of multipliers (ADMM) whose convergence is ensured theoretically. The proposed mixture model and its unmixing algorithm are validated on both synthetic and real images showing competitive results regarding the quality of the inference and the computational complexity when compared to the state-of-the-art algorithms.

This paper presents a new Bayesian model and associated algorithm for depth and intensity profiling using full waveforms from time-correlated single-photon counting (TCSPC) measurements when the photon count in very low. The model represents each Lidar waveform as an unknown constant background level, which is combined in the presence of a target, to a known impulse response weighted by the target intensity and finally corrupted by Poisson noise. The joint target detection and depth imaging problem is expressed as a pixel-wise model selection problem which is solved using Bayesian inference. A Reversible Jump Markov chain Monte Carlo algorithm is proposed to compute the Bayesian estimates of interest. Finally, the benefits of the methodology are demonstrated through a series of experiments using real data.

This paper presents a new Bayesian model and algorithm used for depth and intensity profiling using full waveforms from the time-correlated single photon counting (TCSPC) measurement in the limit of very low photon counts. The model proposed represents each Lidar waveform as a combination of a known impulse response, weighted by the target intensity, and an unknown constant background, corrupted by Poisson noise. Prior knowledge about the problem is embedded in a hierarchical model that describes the dependence structure between the model parameters and their constraints. In particular, a gamma Markov random field (MRF) is used to model the joint distribution of the target intensity, and a second MRF is used to model the distribution of the target depth, which are both expected to exhibit significant spatial correlations. An adaptive Markov chain Monte Carlo algorithm is then proposed to compute the Bayesian estimates of interest and perform Bayesian inference. This algorithm is equipped with a stochastic optimization adaptation mechanism that automatically adjusts the parameters of the MRFs by maximum marginal likelihood estimation. Finally, the benefits of the proposed methodology are demonstrated through a serie of experiments using real data.

Multispectral light detection and ranging (LiDAR) has the potential to recover structural and physiological data from arboreal samples and, by extension, from forest canopies when deployed on aerial or space platforms. In this paper, we describe the design and evaluation of a prototype multispectral LiDAR system and demonstrate the measurement of leaf and bark area and abundance profiles using a series of experiments on tree samples “viewed from above” by tilting living conifers such that the apex is directed on the viewing axis. As the complete recovery of all structural and physiological parameters is ill posed with a restricted set of four wavelengths, we used leaf and bark spectra measured in the laboratory to constrain parameter inversion by an extended reversible jump Markov chain Monte Carlo algorithm. However, we also show in a separate experiment how the multispectral LiDAR can recover directly a profile of Normalized Difference Vegetation Index (NDVI), which is verified against the laboratory spectral measurements. Our work shows the potential of multispectral LiDAR to recover both structural and physiological data and also highlights the fine spatial resolution that can be achieved with time-correlated single-photon counting.

In spite of the rapid growth in the application of active 3D sensing technolo-gies, it has proved difficult to acquire surface data from transparent objects due to the high transmittance and specularity of the surfaces and the con-fusion caused by multiple returns. In this paper, we describe how active time-of-flight range sensing using photon counting can be used to acquire data from several transparent surfaces simultaneously. Following a short introduction, the paper is organised in three main sec-tions. First we review the basic principles of time of flight ranging using time correlated single photon counting. Second, we present some basic the-ory of how data is acquired from multiple surfaces. Third, we present some initial measurements on common transparent objects. Finally, we summarise progress and the necessity for further work. 1