Structure from Motion (SfM) photogrammetry, in conjunction with the Multi-View Stereo (MVS) technique, collectively known as SfM-MVS, emerges as a cost-effective solution for reconstructing 3D structures in real-world environments through the utilization of overlapping images. While SfM photogrammetry finds widespread use in remote sensing applications, challenges persist regarding reconstruction quality, scene segmentation, and computational complexity and efficiency. This paper introduces a workflow wherein semantically segmented images guide the SfM-MVS processing of overlapping images for reconstruction. The proposed workflow is applied to address two challenging tasks. The first task focuses on reconstructing a narrow pier situated over dynamic open ocean waves, while the second task involves simultaneous reconstruction and scene (point cloud) segmentation. Semantic labels assigned to pixels play a crucial role in determining the inclusion or exclusion of specific pixel sets during SfM-MVS processing. This experimental study underscores the promising potential of the proposed workflow to seamlessly integrate with the conventional SfM-MVS processing workflow. The approach not only augments the reconstruction quality in challenging environments but also advances the level of automation in generating spatial products within established SfM photogrammetry software suites. These findings contribute to the ongoing discourse on improving SfM-MVS methodologies for enhanced reconstruction outcomes and increased efficiency in spatial product generation.

Uncrewed aircraft system (UAS)-based surveying offers an efficient way to produce dense point clouds of roadway corridors within the right-of-way (ROW). Common techniques include structure-from-motion and multi-view stereo (SfM/MVS) photogrammetry, or UAS-SfM, and UAS-based light detection and ranging (lidar), or UAS-Lidar. However, considerations such as measurement fidelity and post-processing workflows are necessary to effectively deploy these technologies. This study examines UAS-SfM and UAS-Lidar survey repeatability of a roadway surface by comparing direct georeferencing solutions with and without the use of a ground control point (GCP) network. Field tests examine differences in vertical accuracy and compare differences in digital terrain model (DTM)-based change detection of road-way surface elevation. Repeat UAS-SfM and UAS-Lidar flights were conducted over a flat runway surface acting as a proxy for a typical state highway roadway corridor. The UAS-SfM surveys were conducted with a platform equipped with a 42 MP RGB digital camera and a post-processed kinematic (PPK) global navigation satellite system (GNSS) receiver for accurate image geopositioning. The UAS-Lidar surveys were conducted using a geodetic-grade RIEGL VUX-1LR long-range scanner and a Livox Avia mapping-grade scanner. Direct georeferencing solutions resulted in vertical change detection errors (i.e., root mean square errors) within 2.9 cm for UAS-SfM and between 1.6 cm and 1.8 cm for UAS-Lidar depending on the lidar sensor. The inclusion of GCPs improved UAS-SfM change detection error to within 2.3 cm while UAS-Lidar improved to 0.9 cm for the survey-grade VUX sensor and degraded to 3.1 cm for the Avia sensor.

Structure from Motion (SfM) photogrammetry in combination with the Multi-View Stereo (MVS) technique (SfM-MVS ) is a low-cost and effective tool to reconstruct the 3D structure of the real-world environments and objects using a set of overlapping images. This is approach is often implemented using imagery collected by a small uncrewed aircraft system (UAS) equipped with a high-resolution digital camera (referred to as UAS-SfM). The generated dense point cloud is typically considered the ultimate product of the workflow in almost all commercial and open-source software packages. Although, currently, UAS photogrammetry is being widely used in many remote sensing (RS) applications, there are some challenges regarding the efficiency, quality, and computational complexity in deriving some geospatial products in certain environments, especially where frequent mapping of the area is required to monitor certain changes in the surveyed environment. Efficient mapping of the coastal environments can be challenging due to the presence of water or other moving objects in almost all UAS images. It may also be challenging due to the need for accurate classified 2D and 3D land cover maps or digital terrain models (DTMs) which are required to examine the rate of changes in those environments. Providing those products through the SfM-MVS photogrammetry requires processing a large number of UAS images as well as performing expensive and complicated processing on 2D images and/or 3D point clouds which leads to an exponential increase in the computational complexity and cost in coastal mapping. This study proposes a novel approach that integrates semantic UAS image segmentation into the SfM-MVS photogrammetry workflow to address the aforementioned challenges in UAS photogrammetry mapping in coastal environments. The proposed approach leads to a higher level of automation in generating geospatial products with efficient exploitation of the available computation resources for SfM-MVS computations. The proposed approach exploits state-of-the-art deep learning (DL) for semantic UAS image segmentation to predict land cover image masks. Image masks predicted for moving objects, present in the study area, e.g., water bodies, are used to automatically exclude those areas from SfM-MVS computation. That leads to a higher computation efficiency and an lower noise in the reconstructed model. The proposed approach is then expanded to predict image mask through the DL-based image segmentation model for an automatic generation of the classified point cloud simultaneous with the SfM-MVS processing for reconstruction.

Uncrewed aircraft system (UAS)-based structure-from-motion / multi-view stereo (SfM/MVS) photogrammetry (referred to as UAS-SfM) provides an efficient means for dense 3D mapping of terrain and land cover at local geographic scales. For geospatial applications, UAS-acquired imagery must be accurately georeferenced before the desired mapping products are generated. The traditional method entails a network of precisely surveyed ground control points; referred to as indirect georeferencing. Though accurate, this process is tedious and unsuitable for UAS surveying of some shorelines and remote areas. This study examines the accuracy and repeatability of three different global navigation satellite system (GNSS) solutions for direct georeferencing of UAS imagery: real-time kinematic (RTK), post-processed kinematic (PPK), and precise point positioning (PPP). Specifically, the study evaluates whether these techniques can achieve a UAS-SfM point cloud vertical positional accuracy within 50 cm at the 95% confidence level for shoreline mapping and charting purposes. UAS-SfM field experiments relying on RTK and PPK consistently yielded root mean square errors within 10 cm, whereas PPP showed promising results but was unable to meet the accuracy requirements. Evaluation of baseline distance and sample rate on PPK solutions showed that the best accuracies are achievable when using base stations located within 30 km from the survey site, and the PPK fix percentage is influenced by the base station observation rate.

Information derived from full-waveform (FW) data collected by FW laser scanning systems has already been shown to be relevant for point cloud analysis tasks. Relevant waveform attributes to populate the corresponding point’s feature vector are typically provided through a post-processing FW analysis (FWA) technique based on fitting the echo waveform with a parametric function describing the shape and location of the echo pulse in the waveform. Samples of the digitized echo are the primary source for any waveform analysis using parametric functions. On the other hand, for some FW laser scanning systems, describing the complex system response model using a simple parametric function seems challenging or impractical. Earlier studies have shown the potential of waveform’s digital samples as relevant waveform attributes, for point cloud classification. The main goal of this study is to extend earlier experiments on direct exploitation of returned waveform signals collected by a FW terrestrial laser scanning (TLS) system in a built environment for point cloud classification, to multi-return waveform signals. Furthermore, the classification performance on feature vectors containing calibrated waveform attributes, derived from a waveform processing approach performed in real-time by the FW TLS system, is evaluated on multiple-echo waveforms and compared with the classification performance derived from the proposed FW data classification technique. Classification performance derived through the proposed technique demonstrates high information content of raw digitized waveform samples. Results show that feature vectors containing samples of digitized echoes carry more information about physical properties of the target than those containing calibrated waveform attributes.

Full-waveform (FW) data collected by FW light detection and ranging (lidar) system have long been used to enhance the characterization of the surveyed area. It has already been shown that attributes describing the shape of the echo sig-nal in the feature vector of lidar points improves the performance of land cover classification. In this experiment, rather than fitting the echo waveform with an appropriate parametric model, typically a Gaussian, for feature extraction, the feature vector of each illuminated target is populated with the sam-ples of corresponding digitized waveform. Waveform data is collected over a built environment including both natural and built objects using a full-waveform (FW) terrestrial laser scanning (TLS) system. Results show that samples of raw digitized waveform carry useful information about the physi-cal properties of illuminated targets which lead to a relatively high performance in a multi -class TLS point cloud classifi-cation, where discriminative waveform features are extracted automatically within a proposed convolutional neural network (CNN) model.

In this study, the potential of raw samples of digitized echo waveforms collected by full-waveform (FW) terrestrial laser scanning (TLS) for point cloud classification is investigated. Two different TLS systems are employed, both equipped with a waveform digitizer for access to the raw waveform and online waveform processing which assigns calibrated waveform attributes to each point measurement. Point cloud classification based on samples of the raw single-peak echo waveform is compared with point cloud classification based on the calibrated online waveform attributes. A deep convolutional neural network (DCNN) is designed for the supervised classification. Random forest classifier is used as a benchmark to evaluate the performance of the proposed DCNN model. In addition, feature importance and temporal stability of the raw waveform samples versus the calibrated waveform attributes for point cloud classification are reported. Classification results are evaluated at two study sites, a built environment on a university campus and a coastal wetland environment. Results show that direct classification of the raw waveform samples outperforms classification based on the set of waveform attributes at both study sites. Results also show that the contribution of the range, as the only geometric attribute in the raw waveform feature vector, significantly increases the classification performance. Finally, the performance of the DCNN for filtering ground points to generate a digital terrain model (DTM) based on classification of the raw waveform samples is assessed and compared to a DTM generated from a progressive morphological filter and to real-time kinematic (RTK) GNSS survey data.

Full-waveform analysis (FWA) through modeling and decomposition of the digitized echo waveform, measured by a full-waveform (FW) laser scanning system, is typically employed to derive the waveform attributes, including the number of echoes, amplitude and width of each detected echo in the backscattered waveform signal. It has already been shown that such attributes in the feature vector of each measured point can enhance the performance of semantic point cloud segmentation. In this experiment, however, rather than modeling the waveform, the feature vector of each measured target is populated with the raw samples of the waveform related to the target. Random forest classification is used to classify a 3D scene consisting of both natural and man-made targets. The results show that the raw samples of the digitized waveform are discriminative enough to be used as independent features for a multi-class classification task, where, in this experiment, the overall accuracy of 90% was achieved for classifying the$3D$scene.