This study presents a first-of-its-kind comprehensive data fusion approach combining hyperspectral microwave (HMW) with backscatter lidar (BSL) measurements for improved atmospheric thermodynamic sounding, with particular emphasis on the Earth’s Planetary Boundary Layer (PBL). This is a simulation-based trade study to demonstrate the enhancement of HMW over traditional microwave (MW) only measurements and the additional benefits of incorporating BSL with both approaches. This pioneering HMW+BSL fusion methodology represents a major advancement, achieving superior performance compared to traditional thermodynamic remote sensing approaches. Specifically, this configuration demonstrates significant enhancement in PBL temperature bias vertical stability and reduces standard deviation error (SDV) by 30% compared to traditional MW-only performance. Water vapor retrievals show similar improvements, with SDV reductions of 50% in the PBL and bias values consistently maintained below the 10% requirement threshold of the PBL DSI program, compared to PoR errors exceeding 30% bias in challenging cloudy regimes. Case studies across diverse oceanic regions reveal particular advantages of this data fusion approach in complex atmospheric conditions, especially in regions dominated by marine stratocumulus clouds and strong temperature inversions where conventional passive-only retrievals are challenging. Beyond thermodynamic profile improvements, our analysis demonstrates remarkable advances in the detection of PBL height (PBLH), with the HMW+BSL configuration achieving mean absolute errors within the 100 meter requirement threshold of the PBL DSI program, representing a step-change improvement over passive-only approaches. This work directly addresses observational gaps identified in the 2017 Earth Science Decadal Survey, positioning our integrated sensing approach as both a near-term enhancement to existing Earth observation capabilities and a pathfinder for future PBL mission architectures.

We present an overview of the 2024 West-Coast Hyperspectral Microwave Sensor Intensive Experiment (WHyMSIE). WHyMSIE is a joint NASA-NOAA multi-sensor airborne experiment, embracing passive and active sensors from the Program of Record (PoR) along with novel technology funded through the NASA ESTO Instrument Incubation Program. At the core of this effort is the demonstration of the Conical Scanning Millimeter-wave Imaging Radiometer Hyperspectral (CoSMIR-H) instrument, a PBL DSI funded effort to develop hyperspectral sounding capability in the thermal microwave domain finalized to improved temperature and water vapor soundings in the Earth’s Planetary Boundary Layer (PBL). An overview of the field campaign design, instrument payload and validation plan is presented here.

Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for LiDAR sensing of Earth. It is based on NASA’s adaptive wavelength scanning LiDAR (AWSL) system. Unlike conventional 1D LiDAR methods, CS-LiDAR utilizes sparse coded laser illumination across a 2D field-of-view. The aim is to compressively capture Earth from hundreds of kilometers above, enabling computational 3D imagery reconstruction with resolution that is comparable to that attained with data collected from just hundreds of meters. The forward imaging model captures the light propagation phenomena affecting the photon pulses transmitted from the sensor to the Earth’s surface and back. This work enhances CS-LiDAR by integrating imaging spectroscopy into a multimodal system and employing a transformer network for the inverse imaging problem, driven by multimodal attention mechanisms. Emulations enabled by enormous observational LiDAR data of Earth, available from NASA’s G-LiHT imaging observatory, highlight the efficacy of methods developed.

This paper proposes an algorithm to enhance the resolution of satellite lidar data using Generative Adversarial Networks (GANs) under the hyperheight data cube framework. A super-resolution algorithm based on adversarial training is applied to overcome the challenges of long-range satellite lidar systems. The algorithm generates high-resolution super-resolved outputs from low-resolution inputs, improving the quality of several lidar representations such as canopy height models and profiles. This approach not only advances lidar-based models but also facilitates sophisticated lidar data analysis for various fields, such as environmental science, urban planning, and disaster management. The super-resolved lidar data provides a more precise depiction of the Earth's surface, opening up new avenues for research and applications in different domains. The framework's effectiveness was validated in the Florida Everglades National Park, where the resolution was increased from a 3m x 6m grid with 10m footprints to a 3m x 3m grid with 3m footprints, and the vertical resolution was enhanced from 0.5m to 0.25m.

We present the design and performance of a Concurrent Artificially-intelligent Spectrometry and Adaptive Lidar System (CASALS) for 3D imaging from Space. With a single fast wavelength tuning laser, CASALS accomplishes a 1,200 resolvable spots swath mapping by grating dispersion wavelength steering. Any subset of these 1,200 spots can be selected by wavelength switching. The validation operating principle was accomplished and reported in IGASS-2022. With configurable base design, we report the designs and progress of the CASALS airplane campaign with 256 contiguous ground spots. It is accomplished with a single fast tuning lase at 1040-nm, 1.152MHz tuning rate, and pulse modulated 2-ns on each wavelength. Return pulses are mapped to an eight-pixel detector array with single-photon sensitivity. The lidar returns are time-multiplexed to two outputs that are digitized with two 1-GSPS-digitizer. A grating spectrometer rejects solar background noise spatially and spectrally. We are developing a 1040-nm CASALS intended for multiple LEO orbit missions: Earth Venture Mission on ESPA Grande, STV Mission on ESPA Grande SmallSat, STV Mission on spacecraft equivalent to ICESat-2.

Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for LiDAR sensing of Earth. It is based on NASA’s adaptive wavelength scanning LiDAR (AWSL) system. Rather than measuring 1D line footprints over a satellite’s swath path as is the norm today, CS-LiDAR adopts sparse coded laser illumination over a 2D wide field-of-view. The objective is to compressively sense Earth from hundreds of km above Earth to then computationally reconstruct the 3D imagery with resolution and coverage as if the data was collected from just hundreds of meters in height. The forward imaging model captures the light propagation phenomena affecting the photon pulses transmitted from the sensor to the Earth’s surface and back. This paper advances CS-LiDAR on many fronts. First, imaging spectroscopy side-information, often jointly available with LiDARs, is integrated into a multimodal imaging system. Secondly, the inverse imaging problem is cast under a transformer network architecture driven by multimodal attention mechanisms. Finally, by directing the snapshot spectral cameras in front of the LiDAR, the transformer mechanisms autonomously adjust the LiDAR’s beam scanning to focus on specific target locations, thus attaining end-to-end optimal adaptive sampling that can respond to varying observational conditions, surface events, and scientific priorities. Emulations enabled by enormous observational LiDAR data of Earth, available from NASA’s G-LiHT imaging observatory, show the advantages attained by methods developed in this work.

LiDAR remote sensing systems are deployed in various platforms including satellites, airplanes, and drones — which, in essence, determines the sampling characteristics of the underlying imaging system. Low-altitude LiDARs provide high photon count and high spatial resolution but only in very localized patches. Satellite LiDARs, on the other hand, provide measurements at a global scale but are limited by low photon count and their samples are sparsely apart along swath line trajectories that are far in between. This paper describes a new class of satellite remote sensing LiDARs, aimed at overcoming the limitations of current satellite imaging systems. It exploits the principles of compressive sensing and machine learning (ML) to compressively sense Earth from hundreds of km above Earth to then reconstruct the 3D imagery with resolution and coverage, as if the data was collected from airborne platforms at just hundreds of meters in height.We introduce a novel representation of waveform altimetry profiles, coined HyperHeight Data Cubes (HHDC), which encompass rich information about the 3D structure of a scene. Canopy height models, digital terrain models, and many other features of a scene that are embedded in HHDC are easily extracted with simple statistical quantiles.We introduce machine learning methods to reconstruct the compressive LiDAR measurements so as to attain high-resolution, dense coverage, and broad field-of-view per swath pass. ML training data is attained from NASA’s G-LiHT imaging missions. Simulations with various types of forests across the US illustrate the power of the new LiDAR imaging systems.

We present a comprehensive Earth Planetary Boundary Layer temperature and water vapor retrieval improvement demonstration by the use of hyperspectral microwave measurements. Our results indicate that the use of a hyperspectral sampling in the oxygen and water vapor sounding lines alone provides significant improvements in the lower and free tropospheric thermodynamic fields (up to 40%), when compared against the program of record (i.e., the Advanced Technology Microwave Sounder, ATMS). Our experiments also demonstrate the essential role played by extending the coverage in the so called spectral window regions, leading to an overall PBL temperature and water vapor improvement of up to 50%.

Characterizing the complex three-dimensional (3D) thermodynamic structure of the Planetary Boundary Layer (PBL) from a global perspective remains a challenge. As identified by the 2017 Decadal Survey and the NASA PBL Incubation Study Team Report (STR), enhanced horizontal and vertical resolution in PBL thermodynamic structure and PBL height from space-based sensors will facilitate major advances in Earth System science across a wide array of disciplines. Current Program of Record (POR) space-borne passive sounders (infrared, microwave) were not designed with a specific PBL focus. Consequently, current operational retrieval methods have limitations that preclude them from profiling PBL temperature and water vapor with the requirements expressed in the NASA PBL Incubation Study Team Report (STR). To that end, the report highlights the need for investing in optimal combinations of different remote sensing approaches and technologies spanning the active and passive field. In this framework, the study lists hyperspectral microwave sensors as an "Essential Component" of the future global PBL observing system, to provide accurate PBL and free tropospheric 3D temperature and water vapor structure context to active measurements (e.g., lidars and radars) and in combination with other passive sensors (e.g., infrared and radio occultation).

We have developed a process for evaluating future satellite-based hyperspectral microwave sensor designs using deep neural networks (DNN). Our approach combines a sophisticated simulated data product with a hierarchical deep neural network capable of comparing the relative performance of a variety of different microwave sounder configurations. These configurations include both spectral band coverage and resolution which allows for a thorough investigation of the solution space. The relative performance between these configurations as tested on the prediction of the planetary boundary layer height (PBLH) is used to perform the evaluation. We plan to extend this method to the prediction of entire temperature and water profiles to further refine this process.

New Observing Systems (NOS) will be NASA’s next generation approach for Earth remote sensing, utilizing many diverse observing capabilities to produce optimized measurements integrated from multiple vantage points and in multiple dimensions. NOS will require strong data fusion foundations to be able to intelligently combine, and retrieve information from, data coming from assets differing in characteristics like instrument type, spectral domain, and spatial and temporal resolution. We are developing an end-to-end data fusion framework employing advanced Artificial Intelligence (AI) Machine Learning (ML) techniques with the primary purpose to drive the design and operation of multi-sensor NOS for Earth sciences and beyond. This work requires building ML-enabled analytic tools and advanced environments to take advantage of high-performance computing systems for the creation of a NOS workflow that utilizes large amounts of diverse airborne and satellite observations along with ancillary information including climate and drought time series and soil properties. We are demonstrating the framework using a forest productivity and degradation use case, but the framework is designed to be applicable to a wide variety of NOS scientific objectives.

Low-altitude airborne lidars deliver high spatial resolution swath mapping using dense laser footprint sampling but only in limited areas, while satellite lidars offer global sampling but are hampered by low resolution due to sparse footprints. This work presents a novel approach to satellite lidar remote sensing designed to address the low spatial resolution by leveraging the principles of compressive sensing and machine learning applied to a highly efficient, adaptive lidar capable of dense footprint sampling. Compressive sensing enables the distribution of footprints across a swath with a density appropriate to recover the features of interest, without unnecessarily oversampling the terrain. Machine learning techniques are employed to reconstruct the compressive lidar measurements, leading to high-resolution, dense coverage, and a broad field-of-view per swath pass. HyperHeight Data Cubes are introduced, which offer a wealth of information about the 3D structure of a scene, including digital surface models, canopy height models and the internal organization of canopies. Training data was obtained from NASA's G-LiHT airborne lidar, and simulations of satellite observations performed on various forest types across the US demonstrate the efficacy of the new lidar imaging approach.

We present the design and performance of an adaptive wavelength scanning lidar (AWSL) for highly efficient mapping from low Earth orbit (LEO). Mapping is accomplished by steering a laser beam across 1,200 resolvable spots using wavelength tuning and grating dispersion. Any subset of these 1,200 spots can be selected by wavelength switching. The design is validated with an 1550-nm prototype using a fast wavelength-tuned (500-kHz) pulsed (2-ns) fiber laser with the beam dispersed by gratings for beam steering. Reflected pulses are detected with an eight-pixel detector array with single-photon sensitivity. Eight lidar returns are time-multiplexed to one output that is digitized with a single 1-GSPS-digitizer, to save power. A grating spectrometer rejects solar background noise spatially and spectrally, and images laser footprints on to the detector array. The gratings retain the fiber laser beam quality. We are developing a 1030-nm AWSL intended for a LEO SmallSat platform.

For accurate assessment of vegetation function there is a strong need for advancement in the remote sensing methods and instruments, to enable monitoring of the parameters governing photosynthesis at the temporal frequency relevant to their dynamics and at a spatial scale that allows practical assessment and management. Currently, no single sensor can provide data at the desired high temporal, spectral and spatial resolutions. Our field measurements obtained using the integrated UAS Piccolo system during the summers of2017 and 2018 demonstrate that science quality reflectance and solar induced fluorescence (SIF) data can be retrieved with high temporal frequency using small Unmanned Aerial Systems (UAS). The implemented approach facilitates data comparisons through space and time, and the integration with other spectral satellite and airborne data. This investigation contributes for bridging the gap in Earth observation between field and airborne measurements by implementing autonomous methods for obtaining high temporal frequency spectroscopy measurements from an Unmanned Aerial Systems (UAS) at various illumination conditions. An advancement in the Intelligent Payload Module (IPM) facilitated the implementation of an optimization workflow to collect spectral data for characterizing vegetation reflectance and solar induced fluorescence (SIF).