Multi-spectral imaging is pivotal in numerous industrial, scientific, and medical fields, yet existing high-resolution systems often rely either on bulky prototypes or costly handheld setups. This paper introduces a novel approach to spectral imaging using a cost-effective handheld camera: the Spectral Camera (SpectralCam). It leverages the advanced optics, sensors, and electronics of a standard Canon EOS R100 DSLR (digital single-lens reflex) camera along with a composite 12-color, color-coded aperture (CCA) fabricated with Fuji Velvia 50 film to enhance light polarization into the DSLR. Additionally, we train a denoising diffusion probabilistic model (DDPM) and devise a guided diffusion workflow to reconstruct images across 12 and 24 spectral bands ranging from 430 to 660 nanometers. Notably, our method eliminates the need for application or hardware-specific training by leveraging the capability of generative artificial intelligence (AI), thus allowing for flexible adaptation to various experimental setups. The proposed solution demonstrates the potential to address diverse challenges in multi-spectral imaging by achieving high-resolution spectral data capture, improved adaptability and deployability while significantly reducing costs and complexity.

Sensing the Earth’s surface topography and vegetation (STV) structure is of critical importance for a myriad of scientific applications. STV metrology relies on lidar, radar, stereophotogrammetry, or a combination of these remote sensing techniques. STV metrology, however, suffers from low spatial and height resolution or sparse coverage if lidars and stereophotogrammetry are deployed at orbital heights. Many scientific applications such as bare Earth, cryosphere, and hydrology, require meter or sub-meter STV observables in spatial resolution with submeter vertical resolution. This work aims to overcome the STV resolution gap by using a simple observation system composed of an orbital Compressive Sensing (CS) lidar aided by high-resolution monocular RGB photography. The system first produces a super-resolved digital surface model by fusing satellite CS lidar photon returns with monocular photography using an image-to-image translation generative Brownian Bridge Diffusion Model. Subsequently, the low photon count lidar measurements together with the high-resolution DSM are then used in a constrained Denoising Diffusion Probabilistic Model to reconstruct super-resolved, wall-to-wall, and feature-rich HyperHeight STV Data Cubes. This approach effectively enhances the resolution for satellite LiDAR imagery while reducing generative model hallucinations, thereby improving the reliability and utility of the resulting data products for Earth studies. The achievable spatial resolution depends on the monocular RGB imagery resolution, the photon density of the training point cloud, and the noise level in the LiDAR sensor.

Spaceborne lidars are essential for monitoring Earth’s ecosystems, particularly in imaging forests, glaciers, and natural hazards. However, current satellite lidar systems, such as NASA’s Global Ecosystem Dynamics Investigation (GEDI) and Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2), are limited in spatial resolution and photon density, constraining their ability to capture detailed surface topography and vegetation (STV) 3-D imagery. Airborne systems, such as NASA’s G-LiHT, offer higher resolution but lack global coverage. To address these limitations, compressive satellite lidars (CS-Lidars) have been recently introduced, utilizing coded laser illumination and dynamic wavelength scanning for wide-field 3-D imaging. A novel framework, based on hyperheight data cubes (HHDCs), uses deep learning to transform sparse measurements into 3-D images, but its resolution remains constrained by the physical limitations of the instruments. This article proposes three approaches using generative diffusion models to achieve super-resolution lidar imaging, enhancing satellite data resolution. These methods involve learning conditional probabilities, guiding models via forward imaging, and leveraging high-resolution side information. The results show substantial improvements in the resolution of satellite lidar data, enabling fine-scale studies of forest structure and improving applications in forest management and environmental monitoring. The methodologies were tested in three regions of USA: Florida, Maryland, and California. The models were trained and tested on the first two, and their zero-shot capabilities were tested on the third, showing comparable results.

Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for LiDAR sensing of Earth. It is based on NASA’s adaptive wavelength scanning LiDAR (AWSL) system. Unlike conventional 1D LiDAR methods, CS-LiDAR utilizes sparse coded laser illumination across a 2D field-of-view. The aim is to compressively capture Earth from hundreds of kilometers above, enabling computational 3D imagery reconstruction with resolution that is comparable to that attained with data collected from just hundreds of meters. The forward imaging model captures the light propagation phenomena affecting the photon pulses transmitted from the sensor to the Earth’s surface and back. This work enhances CS-LiDAR by integrating imaging spectroscopy into a multimodal system and employing a transformer network for the inverse imaging problem, driven by multimodal attention mechanisms. Emulations enabled by enormous observational LiDAR data of Earth, available from NASA’s G-LiHT imaging observatory, highlight the efficacy of methods developed.

This paper proposes an algorithm to enhance the resolution of satellite lidar data using Generative Adversarial Networks (GANs) under the hyperheight data cube framework. A super-resolution algorithm based on adversarial training is applied to overcome the challenges of long-range satellite lidar systems. The algorithm generates high-resolution super-resolved outputs from low-resolution inputs, improving the quality of several lidar representations such as canopy height models and profiles. This approach not only advances lidar-based models but also facilitates sophisticated lidar data analysis for various fields, such as environmental science, urban planning, and disaster management. The super-resolved lidar data provides a more precise depiction of the Earth's surface, opening up new avenues for research and applications in different domains. The framework's effectiveness was validated in the Florida Everglades National Park, where the resolution was increased from a 3m x 6m grid with 10m footprints to a 3m x 3m grid with 3m footprints, and the vertical resolution was enhanced from 0.5m to 0.25m.

Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for LiDAR sensing of Earth. It is based on NASA’s adaptive wavelength scanning LiDAR (AWSL) system. Rather than measuring 1D line footprints over a satellite’s swath path as is the norm today, CS-LiDAR adopts sparse coded laser illumination over a 2D wide field-of-view. The objective is to compressively sense Earth from hundreds of km above Earth to then computationally reconstruct the 3D imagery with resolution and coverage as if the data was collected from just hundreds of meters in height. The forward imaging model captures the light propagation phenomena affecting the photon pulses transmitted from the sensor to the Earth’s surface and back. This paper advances CS-LiDAR on many fronts. First, imaging spectroscopy side-information, often jointly available with LiDARs, is integrated into a multimodal imaging system. Secondly, the inverse imaging problem is cast under a transformer network architecture driven by multimodal attention mechanisms. Finally, by directing the snapshot spectral cameras in front of the LiDAR, the transformer mechanisms autonomously adjust the LiDAR’s beam scanning to focus on specific target locations, thus attaining end-to-end optimal adaptive sampling that can respond to varying observational conditions, surface events, and scientific priorities. Emulations enabled by enormous observational LiDAR data of Earth, available from NASA’s G-LiHT imaging observatory, show the advantages attained by methods developed in this work.

LiDAR remote sensing systems are deployed in various platforms including satellites, airplanes, and drones — which, in essence, determines the sampling characteristics of the underlying imaging system. Low-altitude LiDARs provide high photon count and high spatial resolution but only in very localized patches. Satellite LiDARs, on the other hand, provide measurements at a global scale but are limited by low photon count and their samples are sparsely apart along swath line trajectories that are far in between. This paper describes a new class of satellite remote sensing LiDARs, aimed at overcoming the limitations of current satellite imaging systems. It exploits the principles of compressive sensing and machine learning (ML) to compressively sense Earth from hundreds of km above Earth to then reconstruct the 3D imagery with resolution and coverage, as if the data was collected from airborne platforms at just hundreds of meters in height.We introduce a novel representation of waveform altimetry profiles, coined HyperHeight Data Cubes (HHDC), which encompass rich information about the 3D structure of a scene. Canopy height models, digital terrain models, and many other features of a scene that are embedded in HHDC are easily extracted with simple statistical quantiles.We introduce machine learning methods to reconstruct the compressive LiDAR measurements so as to attain high-resolution, dense coverage, and broad field-of-view per swath pass. ML training data is attained from NASA’s G-LiHT imaging missions. Simulations with various types of forests across the US illustrate the power of the new LiDAR imaging systems.

Low-altitude airborne lidars deliver high spatial resolution swath mapping using dense laser footprint sampling but only in limited areas, while satellite lidars offer global sampling but are hampered by low resolution due to sparse footprints. This work presents a novel approach to satellite lidar remote sensing designed to address the low spatial resolution by leveraging the principles of compressive sensing and machine learning applied to a highly efficient, adaptive lidar capable of dense footprint sampling. Compressive sensing enables the distribution of footprints across a swath with a density appropriate to recover the features of interest, without unnecessarily oversampling the terrain. Machine learning techniques are employed to reconstruct the compressive lidar measurements, leading to high-resolution, dense coverage, and a broad field-of-view per swath pass. HyperHeight Data Cubes are introduced, which offer a wealth of information about the 3D structure of a scene, including digital surface models, canopy height models and the internal organization of canopies. Training data was obtained from NASA's G-LiHT airborne lidar, and simulations of satellite observations performed on various forest types across the US demonstrate the efficacy of the new lidar imaging approach.

Coded aperture snapshot spectral imager (CASSI) senses the spectral information of a 2-D scene and captures a set of coded measurement data that can be used to reconstruct the 3-D spatio-spectral datacube of the input scene by compressive sensing algorithms. The coded aperture (CA) in CASSI plays a crucial role in modulating the spatial information. The pixels in CA are typically square, switched binary ON–OFF, and aligned with the pixels of focal plane array (FPA). Instead of this binary modulation, this letter explores a simple yet effective approach to enabling an equivalent grayscale modulation, which can increase the sensing degree of freedom in CASSI systems. In particular, we deliberately introduce misalignment between the CA pixels and the FPA pixels, such that the spatial modulation of one FPA pixel is determined by four adjacent CA pixels instead of one. Numerical experiments show that the proposed equivalent grayscale modulation induced by misalignment can significantly improve the CASSI reconstruction when compared with current methods, whether a random CA or an optimal blue noise CA is used. More importantly, it does not incur in any cost to the CASSI system.