Stereo matching recovers 3D scene information based on the correlation between corresponding pixels. Despite impressive progress, existing methods lack sufficient correlation priors in ill-posed regions such as occlusions, detailed and reflective regions. In this paper, we propose Geometry Aware Stereo Matching Network (GEAStereo) to enhance geometric structure perception and address this issue. We adaptively incorporate the Monocular Disparity Distribution Prior into the stereo cost volume, building Mono-Stereo Fusion Volume (MSFV), which effectively captures global geometric structures and rectifies the correlation information in ill-posed regions. Furthermore, we introduce rich detail information from gradient features and construct a Detail-Aware Volume (DAV) by aggregating the group-wise cost volume under the guidance of gradient spatial attention, thus enhancing the correlation modeling in detailed structures. Jointly, MSFV and DAV provide rich correlation priors for disparity iterative optimization. Experimental results show that our method achieves competitive results on the ETH3D and KITTI2015 benchmarks. Compared with the state-of-the-art methods, our method demonstrates stronger performance in zero-shot generalization.

The vegetation height has been identified as a key biophysical parameter to justify the role of forests in the carbon cycle and ecosystem productivity. Therefore, consistent and large-scale forest height is essential for managing terrestrial ecosystems, mitigating climate change, and preventing biodiversity loss. Since spaceborne multispectral instruments, Light Detection and Ranging (LiDAR), and Synthetic Aperture Radar (SAR) have been widely used for large-scale earth observation for years, this paper explores the possibility of generating largescale and high-accuracy forest heights with the synergy of the Sentinel-1, Sentinel-2, and ICESat-2 data. A Forest Height Generative Adversarial Network (FH-GAN) is developed to retrieve forest height from Sentinel-1 and Sentinel-2 images sparsely supervised by the ICESat-2 data. This model is made up of a cascade forest height and coherence generator, where the output of the forest height generator is fed into the spatial discriminator to regularize spatial details, and the coherence generator is connected to a coherence discriminator to refine the vertical details. A progressive strategy further underpins the generator to boost the accuracy of multi-source forest height estimation. Results indicated that FH-GAN achieves the best RMSE of 2.10 m at a large scale compared with the LVIS reference and the best RMSE of 6.16 m compared with the ICESat-2 reference.

Deep neural networks based on unrolled iterative algorithms have achieved remarkable success in sparse reconstruction applications, such as synthetic aperture radar (SAR) tomographic inversion (TomoSAR). However, the currently available deep learning-based TomoSAR algorithms are limited to 3-D reconstruction. The extension of deep learning-based algorithms to 4-D imaging, i.e., differential TomoSAR (D-TomoSAR) applications, is impeded mainly due to the high-dimensional weight matrices required by the network designed for D-TomoSAR inversion, which typically contain millions of freely trainable parameters. Learning such huge number of weights requires an enormous number of training samples, resulting in a large memory burden and excessive time consumption. To tackle this issue, we propose an efficient and accurate algorithm called HyperLISTA-ABT. The weights in HyperLISTA-ABT are determined in an analytical way according to a minimum coherence criterion, trimming the model down to an ultra-light one with only three hyperparameters. Additionally, HyperLISTA-ABT improves the global thresholding by utilizing an adaptive blockwise thresholding (ABT) scheme, which applies block-coordinate techniques and conducts thresholding in local blocks, so that weak expressions and local features can be retained in the shrinkage step layer by layer. Simulations were performed and demonstrated the effectiveness of our approach, showing that HyperLISTA-ABT achieves superior computational efficiency with no significant performance degradation compared to the state-of-the-art methods. Real data experiments showed that a high-quality 4-D point cloud could be reconstructed over a large area by the proposed HyperLISTA-ABT with affordable computational resources and in a fast time.

In this paper, we proposed a new framework for 3D point cloud simulation for forest above-ground biomass estimation. It takes tree variables as input and automatically generates 30m by 30m scenes and simulates their corresponding Li-DAR point clouds. 2000 3D tree models of 10 species are generated, with which 5000 forest scenes representing four types of eco-regions are built. Their corresponding biomass are then calculated with allometric equations. We used the simulated tropical scenes (1000 samples) to test four classical machine learning models’ ability in biomass estimation from point clouds. Experimental results show that data augmentation is able to significantly boost test accuracy; self-supervised learning can improve the estimation results; among the four models, ResNet18 is the best baseline model which has achieved a R-square score of 0.6615, and reduced the root mean square error to 202 kg (mean biomass in our dataset is 2114kg).

In order to achieve the objective of accurate and reliable use of deep neural networks for Earth Observation in large-scale scene understanding and interpretation, a large and diverse dataset with proper quantification of uncertainty is required. In this work, we exemplify the lack of a benchmark dataset and present the progress of a novel benchmark dataset for uncertainty quantification of deep learning models in the classic problem of building segmentation from overhead imagery. We present a synthetic dataset where synthetic UAV images were rendered from 3D mesh models of Berlin, Germany. The building masks were extracted from precise LoD-2 building models of the same area. We compare and contrast the performances of baseline methods for semantic segmentation and various uncertainty quantification techniques on this dataset. The experiments show that U-Net is the most accurate model with mIoU of 0.812. Moreover, the Bayesian model is found to be the most reliable uncertainty quantification method on our dataset, with the least ECE.

This paper proposes a forest height estimation method based on Seq2Seq models for zero-baseline repeat-pass PolInSAR acquisitions. In zero-baseline configuration, the polarimetric coherence of RMoG model can be refined as a real function varying with ground to volume ratio (polarization) since the phase information is negligible. The primary decorrelation in this real polarimetric coherence is derived from the temporal changes, which has an intuitive connection with forest height and can be further used for forest height estimation. An initial attempt is to estimate the sequence of polarimetric coherence and ground to volume ratio directly from the PolInSAR data and then extract the forest height from the estimated sequence based on the nonlinear functions originating from the RMoG model. However, large discrepancies between the estimated and the model-based sequences make it hard to retrieve the forest height accurately. Thereby, a trained Seq2Seq model is used so that the characteristics of the output sequence can be easily recognized by the model for forest height estimation. Experiments are conducted on PolInSAR data simulated by PolSARproSim+ and results indicate that forest height can be effectively extracted from zero-baseline repeat-pass data with the proposed method.

In this paper, we propose a bounding box regression network for building height retrieval using a single TerraSAR - X stripmap image. The proposed network employs building footprints from GIS data and exploits the location relationship between a building's footprint and its bounding box, enabling fast computation. Experimental results over Rotterdam show that the proposed network can reduce the computation cost significantly while keeping the height accuracy of individual buildings compared to a Faster R-CNN based method.

To achieve super-resolution synthetic aperture radar (SAR) tomography (TomoSAR), compressive sensing (CS)-based algorithms are usually employed, which are, however, computationally expensive, and thus is not often applied in large-scale processing. Recently, deep unfolding techniques have provided a good combination of physical model-based algorithms and the ability of neural networks to learn from data. In this vein, iterative CS-based algorithms can usually be un-rolled as neural networks with only 10 to 20 layers. When trained, it shows great computational efficiency for further TomoSAR processing. However, the learning architecture of neural networks built in this approach tends to result in error propagation and information loss, thus degrading the performance. In this paper, we propose to employ complex-valued sparse long short-term memory (CV-SLSTM) units to tackle this problem by incorporating historically updating information into the optimization procedure and preserving full information. Simulations are carried out to validate the performance of the proposed algorithm.

Finding sparse solutions of underdetermined linear systems commonly requires the solving ofL1regularized least squares minimization problem, which is also known as the basis pursuit denoising (BPDN). They are computationally expensive since they cannot be solved analytically. An emerging technique known asdeep unrollingprovided a good combination of the descriptive ability of neural networks, explainable, and computational efficiency for BPDN. Many unrolled neural networks for BPDN, e.g. learned iterative shrinkage thresholding algorithm and its variants, employ shrinkage functions to prune elements with small magnitude. Through experiments on synthetic aperture radar tomography (TomoSAR), we discover the shrinkage step leads to unavoidable information loss in the dynamics of networks and degrades the performance of the model. We propose a recurrent neural network (RNN) with novel sparse minimal gated units (SMGUs) to solve the information loss issue. The proposed RNN architecture with SMGUs benefits from incorporating historical information into optimization, and thus effectively preserves full information to the final output. Taking TomoSAR inversion as an example, extensive simulations demonstrated that the proposed RNN outperforms the state-of-the-art deep learning-based algorithm in terms of super-resolution power as well as generalization ability. It achieved 10% to 20% higher double scatterers detection rate and is less sensitive to phase and amplitude ratio difference between scatterers. Test on real TerraSAR-X spotlight images also shows high-quality 3-D reconstruction of test site.

Synthetic aperture radar tomography (TomoSAR) has been extensively employed in 3-D reconstruction in dense urban areas using high-resolution SAR acquisitions. Compressive sensing (CS)-based algorithms are generally considered as the state-of-the art in super-resolving TomoSAR, in particular in the single look case. This superior performance comes at the cost of extra computational burdens, because of the sparse reconstruction, which cannot be solved analytically, and we need to employ computationally expensive iterative solvers. In this article, we propose a novel deep learning-based super-resolving TomoSAR inversion approach,$\boldsymbol {\gamma }$-Net, to tackle this challenge.$\boldsymbol {\gamma }$-Net adopts advanced complex-valued learned iterative shrinkage thresholding algorithm (CV-LISTA) to mimic the iterative optimization step in sparse reconstruction. Simulations show the height estimate from a well-trained$\boldsymbol {\gamma }$-Net approaches the Cramér-Rao lower bound (CRLB) while improving the computational efficiency by one to two orders of magnitude comparing to the first-order CS-based methods. It also shows no degradation in the super-resolution power comparing to the state-of-the-art second-order TomoSAR solvers, which are much more computationally expensive than the first-order methods. Specifically,$\boldsymbol {\gamma }$-Net reaches more than 90% detection rate in moderate super-resolving cases at 25 measurements at 6 dB SNR. Moreover, simulation at limited baselines demonstrates that the proposed algorithm outperforms the second-order CS-based method by a fair margin. Test on real TanDEM-X data with just six interferograms also shows high-quality 3-D reconstruction with high-density detected double scatterers.

The recent local climate zones (LCZ) classification scheme provides spatially fine granular descriptions of inner urban morphology. It is universally applicable to cities worldwide and capable of supporting various urban studies. Although optical and dual-pol synthetic aperture radar (SAR) data continue to push the frontiers of this task, the potential of quad-pol SAR data for LCZ classification is not yet explored. In this article, we propose a novel complex-valued convolutional neural network (CNN),SAR4LCZ-Net, to tackle this challenge. SAR4LCZ-Net improves the state-of-the-art by exploiting two facts of this specific task: the semantic hierarchical structure of the LCZ classification scheme and the complex-valued nature of quad-pol SAR data. To validate the performance of our algorithm, we generate a Chinese Gaofen-3 quad-pol SAR dataset for LCZ which covers 31 cities around the world. Results show that the proposed SAR4LCZ-Net improves 2.4% on overall accuracy (OA) and 4.5% on average accuracy (AA) compared with the real-valued CNN with the same structure. Gaofen-3 quad-pol SAR data also showed its advantage over the dual-pol Sentinel-1 data. It enhanced 5.0% on OA and 7.2% on AA in LCZ classification, under a fair comparison with a model trained by Sentinel-1 of the same area.

Synthetic aperture radar tomography (TomoSAR) has been widely employed in 3-D urban mapping. However, state-of-the-art super-resolving TomoSAR algorithms are computationally expensive, because conventional numerical solvers need to solve the$l_{2^{-}}l_{1}$mix norm minimization. This paper proposes a computationally efficient super-resolving To-moSAR inversion algorithm based on deep learning. We studied the potential of deep learning to mimic a conventional$l_{2}-l_{1}$mix norm solver, i.e. iterative shrinkage thresholding algorithm (ISTA), and proposed several improvements of the complex-valued learned ISTA for TomoSAR inversion. Investigation on the super-resolution ability and estimator efficiency of the proposed algorithm shows that the proposed algorithm approaches the Cramer Rao lower bound (CRLB) with a computational efficiency more than 100 times better than the conventional solver.

Interferometric synthetic aperture radar (InSAR) techniques are powerful tool for reconstructing the 3-D position of scatterers, especially for the urban areas. Since the estimation accuracy depends on the inverse of number of interferograms and signal-to-noise ratio (SNR), it is necessary to use as many as possible interferograms in order to achieve more accurate result. However, the number of interferograms of TanDEM-X data is generally limited for most areas. Therefore, in order to maintain the estimation accuracy, one feasible way is to increase the SNR. In this work, we proposed a novel framework, which integrates the non-local procedure into SAR tomography inversion and combines the robust estimation. A large-scale demonstration has been carried out with five TanDEM-X bistatic data, which covers the entire city of Munich, Germany. Quantitative evaluation of the reconstructed result with the LiDAR reference exhibits the standard deviation of the height difference is within two meters, which implies the proposed framework has great potential for high quality large-scale 3-D urban modeling.

SAR tomographic inversion (TomoSAR) has been widely employed for 3-D urban mapping. Existing algorithms are mostly based on an explicit inversion of the SAR imaging model, which are often computationally expensive for large scale processing. This is especially true for compressive sensing-based TomoSAR algorithms. Previous literature showed perspective of using data-driven methods like PCA and kernel PCA to decompose the signal and reduce the computational complexity of parameter inversion. This paper gives a preliminary demonstration of a data-driven TomoSAR method based on sparse Bayesian learning. Experiments on simulated data show the proposed algorithm can provide moderate detection rate and super-resolution power, comparing to the state-of-the-art compressive sensing based algorithms. As the proposed algorithm is purely based on conventional (non-superresolving) estimators, it is much more computationally efficient than compressive sensing based ones. This gives us a perspective of employing it for large scale TomoSAR processing. Experiments on real data will be given in the final paper.

Layover separation has been fundamental to many synthetic aperture radar applications, such as building reconstruction and biomass estimation. Retrieving the scattering profile along the mixed dimension (elevation) is typically solved by inversion of the synthetic aperture radar (SAR) imaging model, a process known as SAR tomography. This article proposes a nonlinear blind scatterer separation method to retrieve the phase centers of the layovered scatterers, avoiding the computationally expensive tomographic inversion. We demonstrate that conventional linear separation methods, for example, principle component analysis (PCA), can only partially separate the scatterers under good conditions. These methods produce systematic phase bias in the retrieved scatterers due to the nonorthogonality of the scatterers' steering vectors, especially when the intensities of the sources are similar or the number of images is low. The proposed method artificially increases the dimensionality of the data using kernel PCA, hence mitigating the aforementioned limitations. In the processing, the proposed method sequentially deflates the covariance matrix using the estimate of the brightest scatterer from kernel PCA. Simulations demonstrate the superior performance of the proposed method over conventional PCA-based methods in various respects. Experiments using TerraSAR-X data show an improvement in height reconstruction accuracy by a factor of one to three, depending on the used number of looks.

Multipass SAR interferometry (InSAR) techniques based on meter-resolution spaceborne SAR satellites, such as TerraSAR-X or COSMO-SkyMed, provide 3D reconstruction and the measurement of ground displacement over large urban areas. Conventional methods such as persistent scatterer interferometry (PSI) usually requires a fairly large SAR image stack (usually in the order of tens) to achieve reliable estimates of these parameters. Recently, low rank property in multipass InSAR data stack was explored and investigated in our previous work (J. Kang et al., “Object-based multipass InSAR via robust low-rank tensor decomposition,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 6, 2018). By exploiting this low rank prior, a more accurate estimation of the geophysical parameters can be achieved, which in turn can effectively reduce the number of interferograms required for a reliable estimation. Based on that, this article proposes a novel tensor decomposition method in a complex domain, which jointly exploits low rank and variational prior of the interferometric phase in InSAR data stacks. Specifically, a total variation (TV) regularized robust low rank tensor decomposition method is exploited for recovering outlier-free InSAR stacks. We demonstrate that the filtered InSAR data stacks can greatly improve the accuracy of geophysical parameters estimated from real data. Moreover, this article demonstrates for the first time in the community that tensor-decomposition-based methods can be beneficial for large-scale urban mapping problems using multipass InSAR. Two TerraSAR-X data stacks with large spatial areas demonstrate the promising performance of the proposed method.

Multibaseline interferometric synthetic aperture radar (InSAR) techniques are effective approaches for retrieving the 3-D information of urban areas. In order to obtain a plausible reconstruction, it is necessary to use more than 20 interferograms. Hence, these methods are commonly not appropriate for large-scale 3-D urban mapping using TanDEM-X data, where only a few acquisitions are available in average for each city. This article proposes a new SAR tomographic processing framework to work with those extremely small stacks, which integrates the nonlocal filtering into SAR tomography inversion. The applicability of the algorithm is demonstrated using a TanDEM-X multibaseline stack with five bistatic interferograms over the whole city of Munich, Germany. A systematic comparison of our result with TanDEM-X raw digital elevation models (DEMs) and airborne LiDAR data shows that the relative height accuracy of two-third buildings is within 2 m, which outperforms the TanDEM-X raw DEM. The promising performance of the proposed algorithm paved the first step toward high-quality large-scale 3-D urban mapping.

The SARptical dataset1consists of about 10,000 matching pairs of very high resolution optical and SAR image patches extracted from TerraSAR-X very high-resolution spotlight images and aerial UltraCAM optical images in dense urban areas of the city of Berlin, Germany. This dataset is distinct from any other existing SAR optical dataset, because the 3D location of the center pixels of the SAR and optical were explicitly matched. Still, creating such dataset poses a fundamental challenge. The reason is that a pixel level matching between SAR and optical images is generally impossible to achieve a without the assistance of a precise 3-D model. This is especially true in dense urban areas, because of the inevitable layover caused by the side-looking SAR imaging geometry. Such misalignment will affect applications like joint classification using SAR and optical images, and raise the difficulty in designing machine learning algorithms. This paper will discuss the challenge of jointly using SAR and optical images for remote sensing applications and propose possible methods to mitigate those misalignment errors.

Hyperspectral images and polarimetric synthetic aperture radar (PolSAR) data are two important data sources, yet they barely appear under the same scope, even though multi-modal data fusion is attracting more and more attention. To our best knowledge, this paper investigates for the first time semi-supervised manifold alignment (SSMA) for the fusion of the hyperspectral image and PolSAR data. The SSMA searches a latent space where different data sources are aligned, which is accomplished by using the label information and the topological structure of the data. This paper is the first attempt to apply topological data analysis (TDA), a recent mathematic sub-field of data analysis, in remote sensing. It aims to reveal relevant information from the shape of a data in its feature space, and has been proven powerful in medicine. The paper also proposes a novel algorithm, MAPPER-regularized manifold alignment, which embeds the TDA into a semi-supervised manifold alignment for the fusion of the hyper-spectral image and PolSAR data. The proposed algorithm exhibits superior performance in fusing a simulated EnMAP data set and a Sentinel-1 data set for an image of Berlin.

Multi-baseline synthetic aperture radar (SAR) interferometric techniques, such as SAR tomography, is well established for 3-D reconstruction in the urban area. These methods usually require fairly large interferometric stacks (> 20 images) for a reliable reconstruction. Hence, they are usually not directly applicable for large-scale 3-D urban mapping using TanDEM-X data where only a few acquisitions are available in average for each city. This work proposes a new SAR tomographic processing framework to those extremely small stacks. The applicability of the algorithm is demonstrated using a TanDEM-X multi-baseline stack with five bistatic interferograms over the whole city of Munich, Germany. Systematic comparison of our result with TanDEM-X raw digital elevation models (DEM) and airborne LiDAR data shows that the relative height accuracy is two meters, which outperforms the TanDEM-X raw DEM. The promising performance of the proposed algorithm paved the first step towards high quality large-scale 3-D urban mapping.

In this paper, we propose a framework for the automatic registration of GIS building footprint polygons to a corresponding SAR image through the corresponding features of building walls in the two data. To extract feature lines, the Potts model is adopted for SAR image segmentation, and visibility test is performed on both data. The feature lines are then sampled to two point sets, and are registered using Iterative Closest Point (ICP) algorithm. The test result shows a registration accuracy of 0.67 m in azimuth direction, and 1.64m in range direction.

This paper addresses the highly challenging problem of automatically detecting man-made structures especially buildings in very high-resolution (VHR) synthetic aperture radar (SAR) images. In this context, this paper has two major contributions. First, it presents a novel and generic workflow that initially classifies the spaceborne SAR tomography (TomoSAR) point clouds-generated by processing VHR SAR image stacks using advanced interferometric techniques known as TomoSAR-into buildings and nonbuildings with the aid of auxiliary information (i.e., either using openly available 2-D building footprints or adopting an optical image classification scheme) and later back project the extracted building points onto the SAR imaging coordinates to produce automatic large-scale benchmark labeled (buildings/nonbuildings) SAR data sets. Second, these labeled data sets (i.e., building masks) have been utilized to construct and train the state-of-the-art deep fully convolution neural networks with an additional conditional random field represented as a recurrent neural network to detect building regions in a single VHR SAR image. Such a cascaded formation has been successfully employed in computer vision and remote sensing fields for optical image classification but, to our knowledge, has not been applied to SAR images. The results of the building detection are illustrated and validated over a TerraSAR-X VHR spotlight SAR image covering approximately 39 km2-almost the whole city of Berlin- with the mean pixel accuracies of around 93.84%.

Obtaining city models in a large scale is usually achieved by means of remote sensing techniques, such as synthetic aperture radar (SAR) interferometry and optical image stereogrammetry. Despite the controlled quality of these products, such observation is restricted by the characteristics of their sensor platform, such as revisit time and spatial resolution. Over the last decade, the rapid development of online geographic information systems, such as Google map, has accumulated vast amount of online images. Despite their uncontrolled quality, these images constitute a set of redundant spatial-temporal observations of our dynamic 3D urban environment. These images contain useful information that can complement the remote sensing data, especially the SAR images. This paper presents a one of the first studies of fusing online street view images and spaceborne SAR images, for the reconstruction of spatial-temporal (hence 4D) city models. We describe a general approach to geometrically combine the information of these two types of images that are nearly impossible to even coregister without a precise 3D city model due to their distinct imaging geometry. It is demonstrated that, one can obtain a new kind of city model that includes high resolution optical texture for better scene understanding and the dynamics of individual buildings up to the precision of millimeter retrieved from SAR interferometry.

During the past decades, multi-pass SAR interferometry (In-SAR) techniques have been developed for retrieving geophysical parameters such as elevation, over large areas. Conventional method such as periodogram usually requires a fairly large SAR image stack (usually in the order of tens), in order to achieve reliable estimates of these parameters. However, when it comes to large-area processing, it is time-consuming and luxury to obtain a sufficient number of SAR images for the reconstruction. In this paper, we demonstrate a novel multi-pass InSAR method for 3D reconstruction using low rank tensor decomposition. By exploiting the low rank prior knowledge in the multi-pass InSAR stack, simulations show that the proposed method can improve the accuracy of elevation estimates by a factor of two, compared to the state-of-the-art InSAR filtering methods, such as SqueeSAR. The capability of the proposed algorithm is also demonstrated on real data using one TanDEM-X InSAR stack of a complex mountainous area.

Modern spaceborne synthetic aperture radar (SAR) sensors, such as TerraSAR-X/TanDEM-X and COSMO-SkyMed, can deliver very high resolution (VHR) data beyond the inherent spatial scales (on the order of 1m) of buildings, constituting invaluable data source for large-scale urban mapping. Processing this VHR data with advanced interferometric techniques, such as SAR tomography (TomoSAR), enables the generation of 3-D (or even 4-D) TomoSAR point clouds from space. In this paper, we present a novel and generic workflow that exploits these TomoSAR point clouds in a way that is capable to automatically produce benchmark annotated (buildings/non-buildings) SAR datasets. These annotated datasets (building masks) have been utilized to construct and train the state-of-the-art deep Fully Convolution Neural Networks with an additional Conditional Random Field represented as a Recurrent Neural Network to detect building regions in a single VHR SAR image. The results of building detection are illustrated and validated over TerraSAR-X VHR spotlight SAR image covering approximately 39 km2- almost the whole city of Berlin - with mean pixel accuracies of around 93.84%.

The joint interpretation of very high resolution SAR and optical images in dense urban area are not trivial due to the distinct imaging geometry of the two types of images. Especially, the inevitable layover caused by the side-looking SAR imaging geometry renders this task even more challenging. Only until recently, the “SARptical” framework [1], [2] proposed a promising solution to tackle this. SARptical can trace individual SAR scatterers in corresponding high-resolution optical images, via rigorous 3-D reconstruction and matching. This paper introduces the SARptical dataset1, which is a dataset of over 10,000 pairs of corresponding SAR, and optical image patches extracted from TerraSAR-X high-resolution spotlight images and aerial UltraCAM optical images. This dataset opens new opportunities of multisensory data analysis. One can analyze the geometry, material, and other properties of the imaged object in both SAR and optical image domain. More advanced applications such as SAR and optical image matching via deep learning [3], [4] is now also possible.

In this letter, we propose a pseudo-siamese convolutional neural network architecture that enables to solve the task of identifying corresponding patches in very high-resolution optical and synthetic aperture radar (SAR) remote sensing imagery. Using eight convolutional layers each in two parallel network streams, a fully connected layer for the fusion of the features learned in each stream, and a loss function based on binary cross entropy, we achieve a one-hot indication if two patches correspond or not. The network is trained and tested on an automatically generated data set that is based on a deterministic alignment of SAR and optical imagery via previously reconstructed and subsequently coregistered 3-D point clouds. The satellite images, from which the patches comprising our data set are extracted, show a complex urban scene containing many elevated objects (i.e., buildings), thus providing one of the most difficult experimental environments. The achieved results show that the network is able to predict corresponding patches with high accuracy, thus indicating great potential for further development toward a generalized multisensor key-point matching procedure.

The most unique advantage of multipass synthetic aperture radar interferometry (InSAR) is the retrieval of long-term geophysical parameters, e.g., linear deformation rates, over large areas. Recently, an object-based multipass InSAR framework has been proposed by Kang, as an alternative to the typical single-pixel methods, e.g., persistent scatterer interferometry (PSI), or pixel-cluster-based methods, e.g., SqueeSAR. This enables the exploitation of inherent properties of InSAR phase stacks on an object level. As a follow-on, this paper investigates the inherent low rank property of such phase tensors and proposes a Robust Multipass InSAR technique via Object-based low rank tensor decomposition. We demonstrate that the filtered InSAR phase stacks can improve the accuracy of geophysical parameters estimated via conventional multipass InSAR techniques, e.g., PSI, by a factor of 10-30 in typical settings. The proposed method is particularly effective against outliers, such as pixels with unmodeled phases. These merits, in turn, can effectively reduce the number of images required for a reliable estimation. The promising performance of the proposed method is demonstrated using high-resolution TerraSAR-X image stacks.

Polarimetric Synthetic Aperture Radar (PolSAR) is a valuable remote sensing data source. It is usually challenging to interpret PolSAR data, especially in urban areas, and hense, spatial clustering comes as a powerful tool for the application of PolSAR data. In data clustering, similarity measurement indexes are of great importance. By far, there are quite some similarity measures of PolSAR data. However, to our knowledge, there has no practical and systematic evaluation of the performances of these measures. In this paper, we evaluate seven different similarity measurements of PolSAR data in the context of clustering using the conventional spectral clustering algorithm.

One of the most unique benefits of multi-baseline synthetic aperture radar interferometry (InSAR) is the long-term monitoring of subtle ground deformation over large areas. Most state-of-the-art algorithms for retrieving such parameter are based on single pixels, e.g. Permanent Scatterer InSAR [1] or clusters of ergodic pixels with stationary phases e.g. SqueeSAR [2]. None of the studies has addressed the joint inversion in an object level, where the true interferometric phase may be varying subject to topography and deformation. Recently, one study has investigated SAR and optical data fusion in order to make use of the rich semantic information from optical images [3]. Based on that work, we seek to investigate the possibility of an object-level multi-baseline InSAR deformation reconstruction given the semantic information from the corresponding optical images. In this paper, we introduced the tensor model for the multi-baseline InSAR inversion and proposed a maximum a posteriori estimator of the deformation parameters by including a spatial prior function in the objective function. Substantial improvement in the deformation estimation is observed in the experiments using both simulated and the real SAR data.

In this paper, we investigate making use of a convolutional neural network (CNN) to solve the task of identifying corresponding patches in very high resolution (VHR) optical and SAR imagery of complicated urban scenery. By doing so, the binary decision function is learnt directly from automatically generated training data and does not resort to any hand-crafted features. First evaluations show great potential for further studies towards a generalized multi-sensor matching procedure.

The side-looking imaging geometry of synthetic aperture radar (SAR) causes inevitable layover in SAR images. Separating the contributions from different scatterers has been the fundamental for many applications. It is typically solved by explicit inversion of the SAR imaging model to retrieve the scattering profile along the mixed dimension (elevation), which is otherwise known as SAR tomography. This paper proposed a robust blind scatterer separation method to demix the layovered scatterers, avoiding the computationally expensive tomographic inversion. We demonstrate that the state-of-the-art principle component decomposition-based methods are heavily influenced by the nonergodicity of the selected samples, especially in urban area, such as point scatterers appearing often on facades. The proposed method is shown to be more robust than the state-of-the-art. Real data example shows that the proposed method outperforms the state-of-the-art by a factor of three in terms of the accuracy of the retrieved phase.

Deformation monitoring by multipass synthetic aperture radar (SAR) interferometry (InSAR) is, so far, the only imaging-based method to assess millimeter-level deformation over large areas from space. Past research mostly focused on the optimal retrieval of deformation parameters on the basis of a single pixel or a pixel cluster. Only until recently, the first demonstration of object-based urban infrastructure monitoring by fusing InSAR and the semantic classification labels derived from optical images was presented by Wanget al.Given such classification labels in the SAR image, we propose a general framework for object-based InSAR parameter retrieval, where the parameters of the whole object are jointly estimated by the inversion of a regularized tensor model instead of pixelwise. Our approach does not assume the stationarity of each sample in the object, which is usually assumed in other pixel cluster-based methods, such as SqueeSAR. In addition, to handle outliers in real data, a robust phase recovery step prior to parameter retrieval is also introduced. In typical settings, the proposed method outperforms the current pixelwise estimators, e.g., periodogram, by a factor of several tens in the accuracy of the linear deformation estimates. Last but not least, for a practical demonstration on bridge monitoring, we present a full workflow of long-term bridge monitoring using the proposed approach.

Using synthetic aperture radar (SAR) interferometry to monitor long-term millimeter-level deformation of urban infrastructures, such as individual buildings and bridges, is an emerging and important field in remote sensing. In the state-of-the-art methods, deformation parameters are retrieved and monitored on a pixel basis solely in the SAR image domain. However, the inevitable side-looking imaging geometry of SAR results in undesired occlusion and layover in urban area, rendering the current method less competent for a semantic-level monitoring of different urban infrastructures. This paper presents a framework of a semantic-level deformation monitoring by linking the precise deformation estimates of SAR interferometry and the semantic classification labels of optical images via a 3-D geometric fusion and semantic texturing. The proposed approach provides the first “SARptical” point cloud of an urban area, which is the SAR tomography point cloud textured with attributes from optical images. This opens a new perspective of InSAR deformation monitoring. Interesting examples on bridge and railway monitoring are demonstrated.

Deformation monitoring by multi-baseline synthetic aperture radar (SAR) interferometry is so far the only imaging-based method to assess millimeter-level deformation over large areas from space. Past research mostly focused on optimal deformation parameters retrieval on a pixel-basis. Only until recently, the first demonstration of object-based urban infrastructures monitoring by fusing SAR interferometry (InSAR) and the semantic classification labels derived from optical images was presented in [1]–[3]. This paper proposes an algorithm for object-based joint InSAR deformation reconstruction using these classification labels. We derive an object-based multi-baseline InSAR reconstruction model, and propose an efficient algorithm for bridge detection in optical images.

Only until recently, it has been demonstrated that absolute localization with centimeter accuracy can be achieved for manually matched Persistent Scatterer (PS)s from TerraSAR-X images acquired from cross-heading geometries [1]. This paper describes an automatic algorithm for absolute localization of natural PSs in SAR images, where the detection of potential PSs is aided by high resolution optical data. As the focus of the study is on urban area, the target detection part relies on identification of lamp posts using template matching. These targets are, most probably, the only ones visible in SAR images acquired from both ascending and descending orbits. Thus, the methodology includes identification of lamp posts from high resolution optical data and retrieves the precise absolute three-dimensional coordinates of the points from corrected TerraSAR-X timing measurements using the stereo SAR method [1]. Preliminary results for a test site in the city of Berlin acquired from TerraSAR-X high resolution spotlight mode are shown.

Multibaseline SAR interferometry may face unmodeled interferometric phase such as unmodeled motion phase and uncompensated atmospheric phase, as well as non-Gaussian statistics in the context of distributed scatterer. We developed the robust InSAR optimization (RIO) [1] framework to systematically tackle these issues. Experiments show that RIO outperforms the current multibaseline InSAR methods in terms of the variance of the phase history parameters estimates for contaminated observations, while still keeping a relative efficiency of 80% for outlier-free observations.

This paper introduces a framework for robust parameter estimation in multipass interferometric synthetic aperture radar (InSAR), such as persistent scatterer interferometry, SAR tomography, small baseline subset, and SqueeSAR. These techniques involve estimation of phase history parameters with or without covariance matrix estimation. Typically, their optimal estimators are derived on the assumption of stationary complex Gaussian-distributed observations. However, their statistical robustness has not been addressed with respect to observations with nonergodic and non-Gaussian multivariate distributions. The proposed robust InSAR optimization (RIO) framework answers two fundamental questions in multipass InSAR: 1) how to optimally treat images with a large phase error, e.g., due to unmolded motion phase, uncompensated atmospheric phase, etc.; and 2) how to estimate the covariance matrix of a non-Gaussian complex InSAR multivariate, particularly those with nonstationary phase signals. For the former question, RIO employs a robust M-estimator to effectively downweight these images; and for the latter, we propose a new method, i.e., the rank M-estimator, which is robust against non-Gaussian distribution. Furthermore, it can work without the assumption of sample stationarity, which is a topic that has not previously been addressed. We demonstrate the advantages of the proposed framework for data with large phase error and heavily tailed distribution, by comparing it with state-of-the-art estimators for persistent and distributed scatterers. Substantial improvement can be achieved in terms of the variance of estimates. The proposed framework can be easily extended to other multipass InSAR techniques, particularly to those where covariance matrix estimation is vital.

This paper presents a step towards a better interpretation of the scattering mechanism of different objects and their deformation histories in SAR interferometry (InSAR). The proposed technique traces individual SAR scatterer in high resolution optical images where their geometries, materials, and other properties can be better analyzed and classified. And hence scatterers of a same object can be analyzed in group, which brings us to a new level of InSAR deformation monitoring.

This letter describes an efficient approach of multidimensional synthetic aperture radar (SAR) imaging for urban mapping. The proposed approach is an integration of tomographic SAR inversion and the well-known persistent scatterer interferometry (PSI). It consists of three steps: first, a global estimation of the topography and motion parameters using efficient algorithms such as PSI; second, a single and double scatterer discrimination step based on the results of the first step; finally, a tomographic SAR inversion, which is performed on the preclassified double scatterers, using the prior knowledge obtained in the first step, retrieving the topography and motion parameters of both scatterers. The proposed approach has been tested on a dozen of TerraSAR-X high-resolution spotlight image stacks. In this letter, examples from Las Vegas and Berlin are presented. The results are comparable with the one obtained by the most computationally expensive tomographic SAR algorithms (e.g., SLIMMER) only and saves computational time by a factor of 50.

This article presents a technique of fusing point clouds from multiple view angles generated using synthetic aperture radar (SAR) tomography. Using TerraSAR-X high resolution spotlight data stacks, one such point has a population of about 2×107points, with a density of around 106points / km2. Such large point population leads to a high computational cost while doing the fusion in 3D space. Therefore, we introduce a feature-based unsupervised technique for point clouds fusion by detecting and matching building contour end points and aligning flat roofs in the two point clouds. The same idea can also be exploited as a general way to evaluate the fusion accuracy of other fusion techniques.

With the availability of meter resolution space-borne SAR systems, urban monitoring using SAR Tomography (TomoSAR) becomes increasingly popular, because of its layover separation capability. However, compared to Persistent Scatterer Interferometry (PSI), TomoSAR applications are much more computationally expensive. This article introduces a TomoSAR processing system for long-term large urban area mapping and monitoring. Two new features were introduced: 1. PSI was integrated into the currently available TomoSAR algorithms (e.g. TSVD, SVD-Wiener, and SL1MMER) to increase the overall computational efficiency; 2. results from multiple view angles were fused to provide full coverage of each building façade. This processing system handles an entire TerraSAR-X high resolution spotlight scene in an affordable time, achieving scatterer density up to 1 million scatterer/km2from a single stack, comparing to 60 thousand to 100 thousand scatterer/km2for PSI.

Measuring the long-term line-of-sight deformation using a multi pass SAR data stack by standard persistent scatterer technique has been explored since the late 1990s. Researches have been continuously conducted on increasing the data coverage at non-PS rich areas. The recently developed SqueeSAR™ technique has validated the potential of extracting useful information from distributed scatterers. With the availability of high resolution TerraSAR-X spotlight data, this technique can benefit greatly from its higher data density and quality. This article presents an algorithm of parameter estimation at distributed scatterers by maximum likelihood estimator in high resolution TS-X spotlight data. Different to SqueeSAR™, this article pays particular attention to the accurate covariance matrix estimation for phase history retrieval on each individual distributed scatterer. Solutions are presented for adaptive sample selection by a different statistical test (Anderson-Darling). An adaptive multi resolution defringe algorithm is introduced to cope with the problem of accurate fringe removal and in turn accurate covariance matrix estimation. And finally maximum likelihood estimator was employed to estimate the model parameters by weighting each measurement according to its coherence. By combining both the persistent scatterers and distributed scatterers, the increase in the capable monitoring area is phenomenal.