Inverse problems in scientific imaging often seek physical characterization of heterogeneous scene materials. The scene is thus represented by physical quantities, such as the density and sizes of particles (microphysics) across a domain. Moreover, the forward image formation model is physical. An important case is that of clouds, where microphysics in three dimensions (3D) dictate the cloud dynamics, lifetime and albedo, with implications to Earth's energy balance, sustainable energy and rainfall. Current methods, however, recover very degenerate representations of microphysics. To enable 3D volumetric recovery of all the required microphysical parameters, we introduce the neural microphysics field (NeMF). It is based on a deep neural network, whose input is multi-view polarization images. NeMF is pre-trained through supervised learning. Training relies on polarized radiative transfer, and noise modeling in polarization-sensitive sensors. The results offer unprecedented recovery, including droplet effective variance. We test NeMF in rigorous simulations and demonstrate it using real-world polarization-image data.

Scattering-based computed tomography (CT) recovers a heterogeneous volumetric scattering medium using images taken from multiple directions. It is a nonlinear problem. Prior art mainly approached it by explicit physics-based optimization of image-fitting, being slow and difficult to scale. Scale is particularly important when the objects constitute large cloud fields, where volumetric recovery is important for climate studies. Besides speed, imaging and recovery need to be flexible, to efficiently handle variable viewing geometries and resolutions. These can be caused by perturbation in camera poses or fusion of data from different types of observational sensors. There is a need for fast variable imaging projection scattering tomography of clouds (VIP-CT). We develop a learning-based solution, using a deep-neural network (DNN) which trains on a large physics-based labeled volumetric dataset. The DNN parameters are oblivious to the domain scale, hence the DNN can work with arbitrarily large domains. VIP-CT offers much better quality than the state of the art. The inference speed and flexibility of VIP-CT make it effectively real-time in the context of spaceborne observations. The paper is the first to demonstrate CT of a real cloud using empirical data directly in a DNN. VIP-CT may offer a model for a learning-based solution to nonlinear CT problems in other scientific domains. Our code is available online.

Spaceborne observations serve as input to models and retrieval algorithms of various atmospheric properties. Specifically, we seek 3D volumetric scattering tomography of clouds. Towards this, cloud-fields are to be imaged simultaneously from multiple directions. The CloudCT project aims to demonstrate this by a formation of ten nanosatellites. Based on this data, scattering tomography will seek the 3D volumetric distribution of cloud microphysical properties. We present constraints and fundamental considerations for spaceborne formation-based cloud tomography. We quantitatively compare visible polarized imagers, visible unpolarized imagers, and short-wave infra-red unpolarized imagers. Each possibility is considered using a large eddy simulation of clouds. We study tomographic quality in the presence of sensor and photon noise, calibration errors and stray light. We find that a polarized imager of a red waveband is preferable.

We introduce a comprehensive method for space-borne 3D volumetric scattering-tomography of cloud micro-physics, developed for the CloudCT mission. The retrieved micro-physical properties are the liquid-water-content and effective droplet radius within a cloud. We include a model for a perspective polarization imager, and an assumption of 3D variation of there. Elements of our work include computed tomography initialization by a parametric horizontally-uniform micro-physical model. This results in smaller errors than the prior art. The mean absolute errors of the retrieved liquid-water-content and effective-radius are reduced from 62% and 28% to 40% and 9%, respectively. The parameters of this initialization are determined by a grid search of a cost function. Furthermore, we add viewpoints in the cloudbow region, to better sample the polarized scattering phase function. The suggested advances are evaluated by retrieval of a set of clouds generated by large-eddy simulations.

We present 3DeepCT, a deep neural network for computed tomography, which performs 3D reconstruction of scattering volumes from multi-view images. The architecture is dictated by the stationary nature of atmospheric cloud fields. The task of volumetric scattering tomography aims at recovering a volume from its 2D projections. This problem has been approached by diverse inverse methods based on signal processing and physics models. However, such techniques are typically iterative, exhibiting a high computational load and a long convergence time. We show that 3DeepCT outperforms physics-based inverse scattering methods, in accuracy, as well as offering orders of magnitude improvement in computational run-time. We further introduce a hybrid model that combines 3DeepCT and physics-based analysis. The resultant hybrid technique enjoys fast inference time and improved recovery performance.

The composition of the atmosphere is significant to our ecosystem. Accordingly, there is a need to sense distributions of atmospheric scatterers such as aerosols and cloud droplets. There is growing interest in recovering these scattering fields in three-dimensions (3D). Even so, current atmospheric observations usually use expensive and unscalable equipment. Moreover, current analysis retrieves partial information (e.g., cloud-base altitudes, water droplet size at cloud tops) based on simplified 1D models. To advance observations and retrievals, we develop a new computational imaging approach for sensing and analyzing the atmosphere, volumetrically. Our approach comprises a ground-based network of cameras. We deployed it in conjunction with additional remote sensing equipment, including a Raman lidar and a sunphotometer, which provide initialization for algorithms and ground truth. The camera network is scalable, low cost, and enables 3D observations in high spatial and temporal resolution. We describe how the system is calibrated to provide absolute radiometric readouts of the light field. Consequently, we describe how to recover the volumetric field of scatterers, using tomography. The tomography process is adapted relative to prior art, to run on large-scale domains and being in-situ within scatterer fields. We empirically demonstrate the feasibility of tomography of clouds, using ground-based data.

To recover the three dimensional (3D) volumetric matter distribution in an object, the object is imaged from multiple directions and locations. Using these images, tomographic computations seek the distribution. When scattering is significant and under constrained irradiance, tomography must explicitly account for off-axis scattering. Furthermore, tomographic recovery must function when imaging is done in-situ, as occurs in medical imaging and ground-based atmospheric sensing. We formulate tomography that handles arbitrary orders of scattering, using a Monte-Carlo model. The model is highly parallelizable in our formulation. This can enable large scale rendering and recovery of volumetric scenes having a large number of variables. We solve stability and conditioning problems that stem from radiative transfer modeling in-situ.