Tree diameter at breast height (DBH) is a key indicator for assessing forest ecosystems, biomass, and resources. Airborne laser scanning enables large-scale DBH inversion with high accuracy and point density. In mixed forests, existing methods can be categorized into regression-after-classification (RAC) and direct-regression-without-classification (DRWC) ones. However, the RAC methods exhibit excessive complexity and may lose common information among species, and the DRWC methods struggle with appropriately handling varying distributions of tree structural data. Therefore, this study proposes a hierarchical multi-experts mixture density network (HMeMD-Net) that integrates RAC and DRWC for accurate DBH inversion. First, an integrated feature optimization strategy is presented to handle complexity in feature construction. Subsequently, a hierarchical mixture of experts (HMoE) structure is designed to extract deep features through synergistic classification-regression task interactions. Innovatively, this paper designs a reinforcement mechanism for information collaboration utilizing Markov random fields. It optimizes information flow and feature representation across tasks. Finally, the model predicts the structural data distributions across tree species through a mixture density network, capturing the data heterogeneity via a Gaussian mixture model. Experimental results on large-scale public datasetsSYSSIFOSSandFOR-instancedemonstrate that HMeMD-Net outperforms existing methods that are commonly used in forestry (R2:0.93/0.88, RMSE:3.35/3.83cm, rRMSE:15.01/15.83%, MAE:2.54/2.82cm, sMAPE:13.79/13.15%), significantly improving the accuracy of DBH inversion and providing an innovative technical pathway for precise assessment and management of forest resources.

Presents corrections to the paper, “HMeMD-Net: A Hybrid Network for Inverting Tree Diameter at Breast Height in Dense Mixed Forests With Airborne Laser Scanning”.

Extracting rocks from Mars rover images using convolutional neural networks is a semantic segmentation problem gaining increasing attention in planetary science and artificial intelligence. However, this task still faces the challenges of inaccurate extraction of rock boundaries and small rocks. An important reason is that the textural features of the Mars rover images are not discriminative enough between the target and the background. In order to obtain 3-D information to enhance the rock extraction without introducing additional data, we designed a dual-task branch network with a Y-shaped encoder-decoder structure. In our network, the primary semantic segmentation task branch is used to decode textural features into semantic features and output rock extraction results; the auxiliary task depth estimation branch decodes textural features into depth features in 3-D, and transmits them to the primary branch through the spatial attention module to enhance the identification ability of semantic features for boundaries and small targets. In addition, another reason impedes accurate Mars rock extraction is the lack of high-quality annotated training datasets. Therefore, we created a dataset containing 6,325 images pairs with corresponding annotations and depth information, SimMars6K. Ablation and comparison experiments based on this dataset and two actual datasets show that our method achieves 83.1% IoU and 90.8F1-score on simulated dataset, and outperforms other methods on actual datasets with 2% improvement in recall. Transfer learning experiment shows that the pre-training on our simulated data can bring up to additional 5% gain on mIoU and 6% gain on mPA for general model.

The separation of woody and foliage components is beneficial in estimating the physical parameters of forests. However, many current methods incur high computational costs and rely on extensive prior knowledge. These methods display weak abilities in generalization for component separation from various LiDAR sensors and tree species. In this paper, a network that combines sparse convolution and transform blocks is proposed for the separation of woody and foliage components in tree point clouds called SPTNet. The sparse convolution block facilitates efficient and effective local feature extraction, while the transformer block offers a solution for the inadequate global feature extraction in sparse convolution blocks. Point feature extraction blocks, called Morphological Detection Coefficient (MDC) and Normal Difference Operator (NDO), were specifically developed to aid in the segmentation task. Distinct adaptive radius strategies are implemented for each geometric feature block to minimize the need for a priori knowledge. Eight different tree species datasets were used to improve methods, including a simulated larch dataset. The other datasets consist of actual trees and comprise seven distinct tree species along with a large tropical tree dataset. Our experimental results demonstrate that our method attains state-of-the-art performance across all datasets. It’s worth mentioning that SPTNet obtains an OA of 94.69% and 89.96% mIoU on the large tropical dataset, which encompasses 15 tree species. Moreover, SPTNet outperforms FWCNN, the current leading branch and leaf separation approach, by 0.43% OA and 0.72% mIoU.

Efficiently and accurately measuring forest structure is of great significance for high-quality assessment of forest resources. Backpack laser scanning (BLS) has become a common device to acquire forest structural information due to its low cost and high time efficiency. However, complex forest environments bring challenges to BLS-based forest mapping, which faces problems with incomplete data and poor mapping accuracy. In this article, we design a disassembly-free dual-scanner BLS system for complete and accurate forest mapping. We first execute a high precision automatic self-calibration of dual laser scanners by means of angle compensation and the fixed rotation angle. Then, a simultaneous localization and mapping (SLAM) framework by combining the natural feature of trees and Inertial Measurement Unit (IMU) measurements is proposed, in which IMU provides priori motion estimation and motion compensation for dual scanners, and the natural feature of the forest is used to correct motion. The proposed method is validated in three small-scale forest plots with size of 0.1 ha. Experimental results show well performance in terms of mapping accuracy, where the mean errors and the root square mean errors are less than 3.0 cm in both horizontal and vertical directions. Our study demonstrates the effectiveness of the proposed strategy and has the potential to perform accurate and complete mapping in understory.

Fine-scale 3-D morphological features of rocks on the Martian surface provide important clues to Mars exploration missions and scientific discoveries. To obtain such information, an automatic approach for high-precision measurement of 3-D morphological features of Martian rocks is proposed in this letter. The approach directly detects 3-D rocks from dense point cloud that is generated by interpolating triangulated irregular network (TIN) model, through a coarse-to-fine process combining cloth simulation filtering (CSF) and connected-component labeling algorithms. Multiple 3-D morphological features are then precisely extracted by modeling rock points and fitting local ground plane. Experimental results demonstrate that the proposed approach provides an effective way for the extraction of complete 3-D rocks and comprehensive morphological features with over 90% accuracy.

Clumping effect denotes the nonrandomness of foliage. It deviates from the random distribution assumption of Beer’s law which is usually applied to leaf area index (LAI) retrieval from large-footprint full-waveform light detection and ranging (LiDAR). Some studies correct for large gaps-induced between-crown clumping, yet ignore the within-crown clumping. The error of LAI caused by these clumping effects and the influence of the forest structure parameters on them have not been quantitatively studied. This study quantified the between-crown, within-crown, and total clumping indices through a theoretical derivation, clarifying the mechanism of clumping; we used airborne LiDAR point clouds data in 11 290 footprints (diameter = 25 m) to estimate these indices in real forests. We found that: 1) the underestimation of LAI caused by directly applying Beer’s law could be up to 93%, and it decreases with fractional crown coverage but increases with crown length and leaf area density; 2) the method of correcting between-crown clumping improves LAI retrieval for cylindrical canopies effectively; however, 3) considerable underestimation (up to 58%) exists if we neglect the within-crown clumping for other canopies, which has not been realized before; and 4) both the between-crown and the within-crown clumping can be the dominant contributor, and the within-crown clumping was greater than the between-crown clumping in 47% of the studied footprints. In the two physically based LAI retrieval methods, Beer’s law has been commonly used due to its simplicity. Pathways to improve future LAI retrieval would be instrument improvement to capture the between-crown gaps and method study to correct the within-crown clumping further.

Occlusion effect, an inherent problem of terrestrial laser scanning (TLS) measurements, limits the potential of TLS data in tree attribute estimation. Multiple scans seek to mitigate this effect to provide enhanced scan completeness. However, the numbers and locations of the scans (i.e., the scan design) are usually determined via a subjective assessment of the tree density, spatial patterns of trees, and attributes to be derived. These could cause suboptimal scan completeness and limit tree attribute estimation. This study proposed an iterative-mode scan design to minimize the occlusion effect. First, we introduced a PoTo index based on visibility analysis to evaluate how many trees can be scanned from a location and to select effective candidates for the optimal TLS location. Second, we introduced a cumulative degree of ring closure (CDRC) to quantify the scan completeness for each candidate and determine the optimal TLS location. The TLS data sets of virtual forests with field-measured and synthetic plot parameter settings were simulated according to iterative- and regular-mode designs by using a Heidelberg light detection and ranging (LiDAR) Operations Simulator (HELIOS). The results demonstrated that an iterative-mode design can improve the scan completeness of trees compared to the regular-mode design. The tree attribute (diameter at breast height (DBH), tree height, stem curve, and crown volume) estimates of the iterative-mode design were less erroneous than those of the regular-mode design (e.g., the root-mean-square error (RMSE) could decrease the stem curve estimation by 38% and the crown volume estimation by 15%). This study suggests that the iterative-mode design can obtain an improved quality of the TLS data, especially for dense stands.

The 3-D information collected from sample plots is significant for forest inventories. Terrestrial laser scanning (TLS) has been demonstrated to be an effective device in data acquisition of forest plots. Although TLS is able to achieve precise measurements, multiple scans are usually necessary to collect more detailed data, which generally requires more time in scan preparation and field data acquisition. In contrast, mobile laser scanning (MLS) is being increasingly utilized in mapping due to its mobility. However, the geometrical peculiarity of forests introduces challenges. In this article, a test backpack-based MLS system, i.e., backpack laser scanning (BLS), is designed for forest plot mapping without a global navigation satellite system/inertial measurement unit (GNSS-IMU) system. To achieve accurate matching, this article proposes to combine the line and point features for calculating transformation, in which the line feature is derived from trunk skeletons. Then, a scan-to-map matching strategy is proposed for correcting positional drift. Finally, this article evaluates the effectiveness and the mapping accuracy of the proposed method in forest sample plots. The experimental results indicate that the proposed method achieves accurate forest plot mapping using the BLS; meanwhile, compared to the existing methods, the proposed method utilizes the geometric attributes of the trees and reaches a lower mapping error, in which the mean errors and the root square mean errors for the horizontal/vertical direction in plots are less than 3 cm.

Estimation of downward shortwave radiation (DSR) is of great importance in global energy budget and climatic modeling. Although various algorithms have been proposed, effective validation methods are absent for rugged terrains due to the lack of rigorous methodology and reliable field measurements. We propose a two-step validation method for rugged terrains based on computer simulations. The first step is to perform point-to-point validation at local scale. Time-series measurements were applied to evaluate a three-dimensional (3-D) radiative transfer model. The second step is to validate the DSR at pixel-scale. A semiempirical model was built up to interpolate and upscale the DSR. Key terrain parameters were weighted by empirical coefficients retrieved from ground-based observations. The optimum number and locations of ground stations were designed by the 3-D radiative transfer model and Monte Carlo method. Four ground stations were selected to upscale the ground-based observations. Additional three ground stations were set up to validate the interpolated results. The upscaled DSR was finally applied to validate the satellite products provided by MODIS and Himawari-8. The results showed that the modeled and observed DSR exhibited good consistency at point scale with correlation coefficients exceeding 0.995. The average error was around 20 W/m2for the interpolated DSR and 10 W/m2for the upscaled DSR in theory. The accuracies of the satellite products were acceptable at most times, with correlation coefficients exceeding 0.94. From an operational point of view, our method has an advantage of using small amount of ground stations to upscale DSR with relatively high accuracy over rugged terrains.

We propose a visual servoing framework for learning to improve grasps of objects. RGB and depth images from grasp attempts are collected using an automated data collection process. The data is then used to train a Grasp Quality Network (GQN) that predicts the outcome of grasps from visual information. A grasp optimization pipeline uses homography models with the trained network to optimize the grasp success rate. We evaluate and compare several algorithms for adjusting the current gripper pose based on the current observation from a gripper-mounted camera to perform visual servoing. Evaluations in both simulated and hardware environments show considerable improvement in grasp robustness with models trained using less than 30K grasp trials. Success rates for grasping novel objects unseen during training increased from 18.5% to 81.0% in simulation, and from 17.8% to 78.0% in the real world.

Accurate and efficient measurement of leaf angle distribution (LAD) is important for characterizing canopy structures and understanding solar radiation regimes within the plant canopy. The main challenge for obtaining LAD is measuring the orientations of individual leaves rapidly and accurately in complex field conditions. In this letter, we propose an efficient and low-cost approach to estimate both leaf zenith and azimuth angles from smartphone photographs by using a structure from motion (SfM) point cloud and pyramid convolutional neural network (PCNN)-based leaf detection. This SfM-PCNN method first detects individual leaves from 2-D photographs by delineating leaf boundaries, while minimizing the influences of interior leaf textures. The segmented image with leaf annotations is then used to partition the 3-D SfM point cloud into leaf clusters, each of which is fit by a plane to calculate the leaf orientation. The method was validated with manual measurements for five plant species with different leaf sizes, leaf shapes, and leaf textures. The accuracy is satisfactory for a leaf-to-leaf comparison over a Euonymus japonicus Thunb. with R-squared values of 0.84 (RMSE = 6.27°) and 0.97 (RMSE = 12.61°) for zenith and azimuth angle estimations, respectively. The method allows researchers to efficiently acquire LADs of different plants with low cost yet high accuracy.

2D face analysis techniques, such as face landmarking, face recognition and face verification, are reasonably dependent on illumination conditions which are usually uncontrolled and unpredictable in the real world. The current massive data-driven approach, e.g., deep learning-based face recognition, requires a huge amount of labeled training face data that hardly cover the infinite lighting variations that can be encountered in real-life applications. An illumination robust preprocessing method thus remains a very interesting but also a significant challenge in reliable face analysis. In this paper we propose a novel model driven approach to improve lighting normalization of face images. Specifically, we propose to build the underlying reflectance model which characterizes interactions between skin surface, lighting source and camera sensor, and elaborate the formation of face color appearance. The proposed illumination processing pipeline enables generation of the Chromaticity Intrinsic Image (CII) in a log chromaticity space which is robust to illumination variations. Moreover, as an advantage over most prevailing methods, a photo-realistic color face image is subsequently reconstructed, which eliminates a wide variety of shadows whilst retaining the color information and identity details. Experimental results under different scenarios and using various face databases show the effectiveness of the proposed approach in dealing with lighting variations, including both soft and hard shadows, in face recognition.

Walking and climbing robots need to plan longterm routes on both horizontal and vertical terrain, but onboard sensors take images from vantage points that provide strongly foreshortened images that cause the appearance of terrain features to vary greatly by distance and viewing angle. This paper presents a convolutional neural network (CNN) method for predicting valid handhold and foothold locations from single RGB+D images taken at arbitrary tilt angles. Experiments show that the method predicts holds more accurately than comparable learning techniques, and that a route planner based on these predictions generates plausible plans for flat ground, stairs, and walls in rock climbing gyms.

The airborne laser scanner (ALS) provides great potential for mapping the leaf area index (LAI) at the landscape scale using grid cell statistics, while its application is restricted by the lack of clumping information, which has been an unsolved issue highlighted for a long time. ALS generally provides an effective LAI because its footprint is too large to capture small gaps to apply traditional ground-based clumping correction methods. Here, we present a grid cell method based on path length distribution model to calculate the clumping-corrected LAI using ALS data without the requirement of additional field measurements. We separated the within- and between-crown areas to consider between-crown clumping, and used the path length distribution as estimated by local canopy height distribution to consider 3-D foliage profile and within-crown clumping. The path length distribution model takes advantage of the 3-D information rather than the gap size distribution, thus avoiding the limitation of large ALS footprint. With the 0.4-m-footprint ALS data, the results are generally promising and a multilevel clumping analysis is consistent with landscape flown. The ALS LAIs of different resolutions are consistent, with a difference of less than 5% from 5- to 250-m resolutions. Due to its consistency and simple configuration, the method provides an opportunity to map the clumping-corrected LAI operationally and strengthens the ability of airborne lidar to monitor vegetation change and validate the satellite product. This grid cell method based on path length distribution is worth further testing and application using more recent laser technology.

Estimation of daily downward shortwave radiation (DSR) is of great importance in global energy budget and climatic modeling. The combination of satellite-based instantaneous measurements and temporal extrapolation models is the most feasible way to capture daily radiation variations at large scales. However, previous studies did not pay enough attention to topographic effects and simple temporal extrapolation methods were applied directly to rugged terrains which cover a large amount of the land surface. This paper, divided into two parts, aims at analyzing the topographic uncertainties of existing models and proposing a better method based on a mountain radiative transfer (MRT) model to calculate daily DSR. As the first part, this paper analyze the spatiotemporal variations of DSR influenced by topographic effects and checks the applicability of three temporal extrapolation methods on cloud-free days. Considering that clouds also have a strong influence on solar radiation, cloud-free days are chosen for targeted analysis of topographic effects on DSR. Three indices, the coefficient of variation, entropy-based dispersion coefficient (CH), and sill of semivariogram, are put forward to give a quantitative description of spatial heterogeneity. Our results show that the topography can dramatically strengthen the spatial heterogeneity of DSR. The index, CH, has an advantage for quantifying spatial heterogeneity as it offers a tradeoff between accuracy and efficiency. Spatial heterogeneity distorts the daily variation of DSR. Application of extrapolation methods in rugged terrains leads to overestimation of daily average DSR up to 60 W/m2 and a maximum 200 W/m2 error of instantaneous DSR on cloud-free days. This paper makes a quantitative analysis of topographic effects under different spatiotemporal conditions, which lays the foundation for developing a new extrapolation method.

Most existing pose-independent Face Recognition (FR) techniques take advantage of 3D model to guarantee the naturalness while normalizing or simulating pose variations. Two nontrivial problems to be tackled are accurate measurement of pose parameters and computational efficiency. In this paper, we introduce an effective and efficient approach to estimate human head pose, which fundamentally ameliorates the performance of 3D aided FR systems. The proposed method works in a progressive way: firstly, a random forest (RF) is constructed utilizing synthesized images derived from 3D models; secondly, the classification result obtained by applying well-trained RF on a probe image is considered as the preliminary pose estimation; finally, this initial pose is transferred to shape-based 3D morphable model (3DMM) aiming at definitive pose normalization. Using such a method, similarity scores between frontal view gallery set and pose-normalized probe set can be computed to predict the identity. Experimental results achieved on the UHDB dataset outperform the ones so far reported. Additionally, it is much less time-consuming than prevailing 3DMM based approaches.

Terrestrial LiDAR systems have received lots of attention on three-dimensional (3D) structure reconstruction for trees, especially on the branches skeleton generation. On this basis, a method is proposed to add leaves structures based on point density by dividing small cube in the canopy to reduce the influence of uneven distribution of point cloud, combining gap fraction model to retrieve leaf area of a tree using terrestrial LiDAR data. It is successfully applied to reconstruct 3D trees using points data simulated by ray tracing algorithm as well as field measured points data. The relative error of leaf area between reconstructed and real structure is less than 0.9%. Meanwhile, the most relative error of directional gap fraction is also less than 4.1%. The experimental results prove that the method has gotten a satisfied consistency on visual sense and quantitative evaluation between the 3D structure reconstructed and real structure.

Terrestrial LiDAR Scanning(TLS) technology can quickly acquire three-dimensional information of forest canopy with high precision. As a new technique of data collection, it has been gradually applied to characterize structural attributes such as plant area densities. This paper presented a ray-tracing method to simulate laser intersection with a single tree and retrieves the plant area index based on gap-fraction model. The simulation model, based on ray-tracing method, was highly dependent on the sensor configuration and the spatial characteristics of the tree examined. Plant area index was retrieved by the gap-fraction model using the simulated point clouds. Given the significant cost and complexity of LiDAR data acquisition, it was necessary to identify the operational parameters to maximize the benefit. Therefore, the factors that might affect the simulation and inversion procedures are discussed extensively. Results showed that the simulation model was capable of predicting what survey configuration would be optimal and facilitating inversion algorithm development.

This paper presents a new generation of scannerless laser radar measurement system, called GLidar in our project. This system does not require the scanning device; as a result, the dimension of the entire measuring system is small, lightweight, and reliability. The proposed scannerless LiDAR system is especially designated for a small civil UAV platform under a low altitude operation. This paper presents the principle of the array LiDAR imaging, mathematical models of calculating the 3-D coordinates of the laser radar footprints with respect to a mapping coordinate system. Some simulated results are presented on the basis of a test field located in Virginia Wytheville, USA. The simulated results demonstrated that the designated new generation of array LiDAR can achieve 0.06–0.10 m in flat area, 0.31–0.62 m in the edge of house, and 1.23–1.78 m in the forested area compared to an existing DSM data.

In remote sensing and agriculture, corn is a common crop which is often studied. In both cases, it is important to measure the geometric structural parameters such as Leaf Area Index (LAI) and Leaf Angle Distribution (LAD). They are useful indicators that affect corn growth. Kinect is a sensor that can be used to get the distance between the object and Kinect itself. It costs little but offers high accuracy. We use Kinect to obtain point clouds of the corn and build a 3D model of the leaves in order to measure structural parameters. The current results show the proposed method is feasible. But more efforts should be made to improve the automation and practically of this method.

This paper presents a portable Multi-Angle Observation System (MAOS) to quickly collect bi-directional reflectance factor (BRF) and directional thermal radiance of land surface along with the spectroradiometer and thermal radiometer. The new system is able to make more than 13 zenith measurements in six minutes at an arbitrary azimuth direction, with the angle-controlling accuracy better than 2°. More observations are sampled in the hot-spot direction. All operations of the MAOS and data-processing are automatically controlled by the computer. Field campaign of winter wheat canopy shows that the MAOS had captured the angular variations of the BRF.

Airborne LiDAR data and optical imagery are two datasets used for 3D building reconstruction. The researchers have developed a variety of modeling method using these two kinds of data. In this paper, we firstly reconstructed the buildings in the test site using the above three kinds of data sources. And then, we compared the results quantitatively. We adopted the primitive-based building reconstruction method to reconstruct the buildings using the two types of data.

This paper gives a brief description of an Airborne Multi-angle Power Line Inspection System (AMPLIS). AMPLIS is composed by 3 CCD cameras, a Position and Orientation System (POS), a stabilized platform, the data collection and control subsystem. It can be equipped on a helicopter and fly along the lines at a speed of about lOOkm/h at a relative height of 100 m over the power lines. AMPLIS is capable of detecting the distance between the power lines and the ground surface with an accuracy of less than 0.5 m. It can automatically find the dangerous objects beneath the lines which can greatly decrease the man power and cost in power line inspection. It has been successfully tested with good performance in Wuhan, China, 2005.

We develop a photogrammetric power line inspection system. Its main objective is to monitor the relative distance between high voltage power line and around objects, and alert if the warning threshold is exceeded. Our photogrammetric power line inspection system generates DSM of the power line passage, which comprises ground surface and ground objects, for example trees and houses, etc. In order to reveal the dangerous regions, where ground objects are too close to the power line, 3D power line information should be extracted at the same time. In order to improve the automation level of extraction, reduce labour costs and human errors, an automatic pole tower and spacer numbering method is proposed. The pole tower automatic numbering is in accordance with GPS position data of the image having pole tower projection, finds the pole tower whose measurement position is closest to it, and the found pole tower's code number is set to pole tower projection. Then a block can be defined by a start pole tower and an end pole tower. The spacer automatic numbering process is limited within a block, and it can be achieved by using epipolar constraint after an aerial triangulation bundle adjustment. The flight experiment result shows the proposed method can give correct code number to pole towers and spacers, and the proper 3D power line information can be obtained by space intersection using found homologous projections.

There has been little investigation for the automatic extraction of power lines from aerial images due to the low resolution of aerial images in the past decades. With increasing aerial photogrammetric technology and sensor technology, it is possible for photogrammetrists to monitor the status of power lines. This letter analyzes the property of imaged power lines and presents an algorithm to automatically extract the power line from aerial images acquired by an aerial digital camera onboard a helicopter. This algorithm first uses a Radon transform to extract line segments of the power line, then uses the grouping method to link each segment, and finally applies the Kalman filter technology to connect the segments into an entire line. We compared our algorithm with the line mask detector method and the ratio line detector, and evaluated their performances. The experimental results demonstrated that our algorithm can successfully extract the power lines from aerial images regardless of background complexity. This presented method has successfully been applied in China National 863 project for power line surveillance, 3-D reconstruction, and modeling.