Interferogram denoising is critical in interferometric synthetic aperture radar (InSAR) data processing, directly impacting the accuracy and reliability of final products. Current data-driven denoising models rely on training samples with noises simulated from certain statistic models. However, such simulations may not fully capture the complexity of real-world data, resulting in degraded performance in practical applications. Here, we introduce a learning method for InSAR phase denoising, marking the first instance of training a model with real-world noise. Our method consists of two phases: (1) model excitation, where the model is initially trained with simulated noise to develop basic denoising capabilities; (2) refinement boosting, an unsupervised, iterative process where the model gradually refines itself using real-world noise through noise extraction, noise purification, data augmentation, and model enhancement. We identified the cost-optimal denoising model by conducting experiments across network architectures of varying complexity, using identical training datasets and experimental settings. Experiments on synthetic interferograms with varying coherence levels show that our method achieves the lowest denoising errors, outperforming state-of-the-art methods such as NL-InSAR and InSAR-BM3D, while maintaining fast inference times comparable to the BoxCar method. Applied to real Sentinel-1 interferograms, our approach demonstrates superior denoising performance, as evidenced by the fewest residues and the smoothest unwrapping results. By analyzing noises simulated from the coherence-guided statistical model and extracted from Sentinel-1 interferograms, we find significant discrepancies between simulated and real-world noise, underscoring the importance of preparing training samples with real-world noise for InSAR data-driven models, e.g., denoising, unwrapping, and other applications.

Although the widely used time-series InSAR technology makes up for the shortcomings of traditional geological investigation, such as small coverage and low efficiency, it cannot achieve rapid and dynamic detection of new potential landslide due to its long data processing time and insensitivity to short-term new displacement. In this paper, a rapid method for automatically detection new potential landslides in wide area is proposed. Phase-gradient processing is performed based on the DInSAR results to automatically detect the potential landslide, in which the influence from geometric distortion, water, noise from low coherence area, and other errors are analyzed and removed. This method was performed in the Maoergai Reservoir Area where many potential landslides newly emerged during the impoundment period as the great water level fluctuations. As a result, 7 potential landslides with continuous deformation and relatively large deformation were detected. The error source was analyzed and removed. In the validation, an overall accuracy of up to 81% was achieved by comparing the results with the manual detection. This method provides a new way for rapid and automatic detection of new displacements in wide-area, especially for the area (e.g. reservoir area) with dynamic and rapid detection needs.

This article proposes deep convolutional neural networks to detect and map localized, rapid subsidence caused by mining activities using time-series Sentinel-1 synthetic aperture radar (SAR) images. A deformation detection network (DDNet) is developed to automatically identify rapidly subsiding areas from wrapped interferograms, and a phase unwrapping network (PUNet) is designed to unwrap the cropped interferogram patches centered on the detected subsiding locations. To train the two networks, interferogram simulation strategies are developed to generate various training samples using the distorted 2-D Gaussian surface and fractal Perlin noises. The performance of the DDNet is verified by simulations on a synthetic dataset with 13 large interferograms, while the PUNet is evaluated by simulations using synthetic datasets with different levels of deformation gradients and noises. Compared with the traditional and deep-learning methods, the PUNet exhibits excellent performance and efficiency in unwrapping interferograms with rapid mining-induced deformation. The proposed networks are further verified by applying them to Shanxi province, China, which is characterized by serious ground subsidence hazards caused by long-term coal mining activities. The time-series deformations of 1344 detected subsidence areas are calculated with the vertical velocities ranging from −19.7 to −254.8 cm/year. The results are validated using the ascending and descending Sentinel-1 Interferograms and an L-band ALOS-2 interferogram covering the same area within the acquisition period, showing highly consistent vertical deformation rates. The proposed strategy and methods introduce deep learning to the time-series interferometric SAR (InSAR) processing chain and may have profound implications on the detection and monitoring of localized mining-induced deformation using InSAR.

Phase unwrapping is a critical step of interferometric synthetic aperture radar processing, and its accuracy directly determines the reliability of subsequent applications. Many phase unwrapping methods have been proposed, most of which assume that the phase has spatial continuity, while decorrelation noise and aliasing fringes invalidate the assumptions, resulting in poor performance of these methods. To obtain more reliable unwrapping results, in this article, a deep convolutional neural network, called a discontinuity estimation network (DENet), is proposed for predicting the probabilities of phase discontinuities in interferograms. The main advantages of DENet are: 1) using branching structure to extract detailed and high-level features separately and retain details while making full use of contextual information; 2) using multichannel input, including interferogram, range/azimuthal phase gradients, and residues map, to provide effective guidance for discontinuity prediction; and 3) using a single network to estimate phase discontinuities in both range and azimuth directions simultaneously. To train the network, a dataset simulation strategy is proposed to generate enough training samples. The strategy considers a variety of phase components, such as terrain-related phase, random deformation, atmospheric turbulence, and noise. The phase discontinuity estimated by DENet is then converted to costs in the minimum cost flow (MCF) solver of the statistical-cost, network-flow algorithm for phase unwrapping (SNAPHU) to obtain the final unwrapped phase. Based on validations of simulated and real interferograms, the proposed method exhibits excellent performance compared to traditional and deep learning unwrapping methods. The proposed method can effectively unwrap large-scale, low-quality interferograms, which is expected to significantly improve the accuracy of synthetic aperture radar interferometry (InSAR) applications.

Phase unwrapping is a crucial step of InSAR, and its reliability directly determines the feasibility of deformation monitoring. However, severe noise and dense fringes often make the existing unwrapping methods fail. In this work, we propose a convolutional neural network DENet for identifying phase discontinuities and design a data set simulation strategy to generate enough training samples. We combine the traditional cost flow method with the output from DENet to achieve more accurate phase unwrapping. Compared with the GAMMA, the root mean square error of the proposed method on the simulated data set is reduced by 46.4%. We also verified the superior performance of the proposed method on real data sets.

Mining induced subsidence seriously damages the ecological environment and may cause casualties. Therefore, the rapid and reliable monitoring is particularly important. However, due to severe noise and dense fringes, traditional InSAR methods often severely underestimate the deformation rate. Here, we propose a new processing flow and develop two deep convolutional neural networks for fast detection and phase unwrapping of local subsidence cones. The proposed method is applied to Datong City, Shanxi Province, which is rich in mining activates. The processing results verify the reliability of the method.

Distributed burst buffers are a promising storage architecture for handling I/O workloads for exascale computing. Their aggregate storage bandwidth grows linearly with system node count. However, although scientific applications can achieve scalable write bandwidth by having each process write to its node-local burst buffer, metadata challenges remain formidable, especially for files shared across many processes. This is due to the need to track and organize file segments across the distributed burst buffers in a global index. Because this global index can be accessed concurrently by thousands or more processes in a scientific application, the scalability of metadata management is a severe performance-limiting factor. In this paper, we propose MetaKV: a key-value store that provides fast and scalable metadata management for HPC metadata workloads on distributed burst buffers. MetaKV complements the functionality of an existing key-value store with specialized metadata services that efficiently handle bursty and concurrent metadata workloads: compressed storage management, supervised block clustering, and log-ring based collective message reduction. Our experiments demonstrate that MetaKV outperforms the state-of-the-art key-value stores by a significant margin. It improves put and get metadata operations by as much as 2.66× and 6.29×, respectively, and the benefits of MetaKV increase with increasing metadata workload demand.

Burst buffers are becoming an indispensable hardware resource on large-scale supercomputers to buffer the bursty I/O from scientific applications. However, there is a lack of software support for burst buffers to be efficiently shared by applications within a batch-submitted job and recycled across different batch jobs. In addition, burst buffers need to cope with a variety of challenging I/O patterns from data-intensive scientific applications. In this study, we have designed an ephemeral Burst Buffer File System (BurstFS) that supports scalable and efficient aggregation of I/O bandwidth from burst buffers while having the same life cycle as a batch-submitted job. BurstFS features several techniques including scalable metadata indexing, co-located I/O delegation, and server-side read clustering and pipelining. Through extensive tuning and analysis, we have validated that BurstFS has accomplished our design objectives, with linear scalability in terms of aggregated I/O bandwidth for parallel writes and reads.

The growing computing power on leadership HPC systems is often accompanied by ever-escalating failure rates. Checkpointing is a common defensive mechanism used by scientific applications for failure recovery. However, directly writing the large and bursty checkpointing dataset to parallel file systems can incur significant I/O contention on storage servers. Such contention in turn degrades bandwidth utilization of storage servers and prolongs the average job I/O time of concurrent applications. Recently burst buffers have been proposed as an intermediate layer to absorb the bursty I/O traffic from compute nodes to storage backend. But an I/O orchestration mechanism is still desirable to efficiently move checkpointing data from burst buffers to storage backend. In this paper, we propose a burst buffer based I/O orchestration framework, named TRIO, to intercept and reshape the bursty writes for better sequential write traffic to storage servers. Meanwhile, TRIO coordinates the flushing orders among concurrent burst buffers to alleviate the contention on storage server. Our experimental results demonstrated that TRIO could efficiently utilize storage bandwidth and reduce the average job I/O time by 37% on average for data-intensive applications in typical checkpointing scenarios.

The growth of computing power on large-scale systems requires commensurate high-bandwidth I/O systems. Many parallel file systems are designed to provide fast sustainable I/O in response to applications' soaring requirements. To meet this need, a novel system is imperative to temporarily buffer the bursty I/O and gradually flush datasets to long-term parallel file systems. In this paper, we introduce the design of BurstMem, a high-performance burst buffer system. BurstMem provides a storage framework with efficient storage and communication management strategies. Our experiments demonstrate that BurstMem is able to speed up the I/O performance of scientific applications by up to 8.5× on leadership computer systems.

SAR interferometry has long been regarded as an effective tool for wide-area topographic mapping in hilly and mountainous areas. However, quality of InSAR DEM product is usually affected by atmospheric disturbances and decorrelation-induced voids, especially for data acquired in repeat-pass mode. In this paper, we proposed an approach for improved topographic mapping by optimal fusion of multi-orbit InSAR DEMs with correction of atmospheric phase screen (APS). An experimental study with highresolution TerraSAR-X and COSMO-SkyMed datasets covering a mountainous area was carried out to demonstrate the effectiveness of the proposed approach. Validation with a reference DEM of scale 1:50,000 indicated that vertical accuracy of the fused DEM can be better than 5 m.

MapReduce is a widely accepted framework for addressing big data challenges. Recently, it has also gained broad attention from scientists at the U.S. leadership computing facilities as a promising solution to process gigantic simulation results. However, conventional high-end computing systems are constructed based on the compute-centric paradigm while big data analytics applications prefer a data-centric paradigm such as MapReduce. This work characterizes the performance impact of key differences between compute- and data-centric paradigms and then provides optimizations to enable a dual-purpose HPC system that can efficiently support conventional HPC applications and new data analytics applications. Using a state-of-the-art MapReduce implementation Spark and the Hyperion system at Lawrence Livermore National Laboratory, we have examined the impact of storage architectures, data locality and task scheduling to the memory-resident MapReduce jobs. Based on our characterization and findings of the performance behaviors, we have introduced two optimization techniques, namely Enhanced Load Balancer and Congestion-Aware Task Dispatching, to improve the performance of Spark applications.

Offset tracking is an important complement to measure large ground displacements in both azimuth and range dimensions where synthetic aperture radar (SAR) interferometry is unfeasible. Subpixel offsets can be obtained by searching for the cross-correlation peak calculated from the match patches uniformly distributed on two SAR images. However, it has its limitations, including redundant computation and incorrect estimations on decorrelated patches. In this letter, we propose a simple strategy that performs offset tracking on detected point-like targets (PT). We first detect image patches within bright PT by using a sinc-like template from a single SAR image and then perform offset tracking on them to obtain the pixel shifts. Compared with the standard method, the application on the 2010 M 7.2 El Mayor-Cucapah earthquake shows that the proposed PT offset tracking can significantly increase the cross-correlation and thus result in both efficiency and reliability improvements.

Synthetic aperture radar (SAR) image coregistration is a key procedure before interferometric SAR (InSAR) time-series analysis can be started. However, many geophysical data sets suffer from severe decorrelation problems due to a variety of reasons, making precise coregistration a nontrivial task. Here, we present a new strategy that uses a pixel-offset series of detected subimage patches dominated by point-like targets (PTs) to improve SAR image coregistrations. First, all potentially coherent image pairs are coregistered in a conventional way. In this step, we propose a coregistration quality index for each image to rank its relative “significance” within the data set and to select a reference image for the SAR data set. Then, a pixel-offset series of detected PTs is made from amplitude maps to improve the geometrical mapping functions. Finally, all images are resampled depending on the pixel offsets calculated from the updated geometrical mapping functions. We used images from a rural region near the North Anatolian Fault in eastern Turkey to test the proposed method, and clear coregistration improvements were found based on amplitude stability. This enhanced the fact that the coregistration strategy should therefore lead to improved InSAR time-series analysis results.

The advance of high-performance computing systems towards exascale will be constrained by the systems' energy consumption levels. Large numbers of processing components, memory, interconnects, and storage components must all be considered to achieve exascale performance within a targeted energy bound. While application-aware power allocation schemes for computing resources are well studied, a portable and scalable budget-constrained power management scheme for scientific applications on exascale systems is still required. Execution activities within scientific applications can be categorized as CPU-bound, I/O-bound and communication-bound. Such activities tend to be clustered into ‘phases’, offering opportunities to manage their power consumption separately. Our experiments have demonstrated that their performance and energy consumption are affected differently by CPU frequency, an opportunity to fine tune CPU frequency for a minimal impact on the total execution time but significant savings on the energy consumption. By exploiting this opportunity, we present a phase-aware hierarchical power management framework that can opportunistically deliver good tradeoffs between system power consumption and application performance under a power budget. Our hierarchical power management framework consists of two main techniques: Phase-Aware CPU Frequency Scaling (PAFS) and opportunistic provisioning for power-constrained performance optimization. We have performed a systematic evaluation using both simulations and representative scientific applications on real systems. Our results show that our techniques can achieve 4.3%–17% better energy efficiency for large-scale scientific applications.

Exascale computing systems are soon to emerge, which will pose great challenges on the huge gap between computing and I/O performance. Many large-scale scientific applications play an important role in our daily life. The huge amounts of data generated by such applications require highly parallel and efficient I/O management policies. In this paper, we adopt a mission-critical scientific application, GEOS-5, as a case to profile and analyze the communication and I/O issues that are preventing applications from fully utilizing the underlying parallel storage systems. Through in-detail architectural and experimental characterization, we observe that current legacy I/O schemes incur significant network communication overheads and are unable to fully parallelize the data access, thus degrading applications' I/O performance and scalability. To address these inefficiencies, we redesign its I/O framework along with a set of parallel I/O techniques to achieve high scalability and performance. Evaluation results on the NASA discover cluster show that our optimization of GEOS- 5 with ADIOS has led to significant performance improvements compared to the original GEOS-5 implementation.

Persistent Scatterer Interferometry (PS-InSAR) is applied to derive displacement information with millimetric precision. Analyzing stable persistent scatterers from a large stack of SAR images,helps to overcome the geometrical and temporal decorrelation, which occur when using differential interferometry. The removal of the topographic phase with an external DEM seems to cause problems. In our experiment, we select three different DEMs: ASTER GDEM, a DEM derived from a digitized topographic map, and SRTM-3 in order to analyze the influence of the input DEMs for PS-InSAR processing in the Three Gorges area. We find that differential interferogram generation is related to the topographic influence for the PS-InSAR processing and different DEMs get us different PS-InSAR results.

By means of the permanent scatterer (PS) technique, repeated spaceborne synthetic aperture radar (SAR) images with relatively low resolution (about 25 m × 5 m for the European Remote Sensing (ERS) and Envisat satellites) can be used to estimate the displacement (1-mm precision) and 3-D location (1-m precision) of targets that show an unchanged electromagnetic signature. The main drawback of the PS technique is the limited spatial density of targets that behave coherently during the whole observation span (hundreds of PSs per square kilometer in urban site and up to few points in vegetated areas). In this paper, we describe a new approach for multitemporal analysis of SAR images that also allows extracting information from partially coherent targets. The basic idea is to loosen the restrictive conditions imposed by the PS technique. The results obtained in different test sites allowed to increase significantly the spatial coverage of the estimate of height and deformation trend, particularly in extraurban areas.

The phase coherence in synthetic aperture radar interferometry is often used in classification algorithms to detect possible temporal changes of the imaged terrain. However, in mountain areas, the interferometric coherence is also sensitive to the slight variations of the acquisition geometry. In this letter, we propose a very simple but effective method to separate the temporal decorrelation from the geometrical one. Assuming the imaged terrain can be modeled as a distributed target, the geometrical coherence can be estimated by exploiting a topographic model and the sensor acquisition parameters. The discrepancy between the geometrical coherence and the observed one can then be ascribed to temporal changes. Moreover, in presence of pointlike targets, the hypothesis of distributed terrain is no longer valid, and higher values of the observed coherence with respect to the synthetic geometrical one can be used to detect such targets. The proposed approach allows then in mountain areas the following conditions: (1) a simple and very fast rough estimation of the temporal coherence, and (2) the identification of pointlike targets using just two images. The method has been applied and tested in the Badong (China) site using European Remote Sensing satellite tandem data.

After the Three Gorges Dam began to function in China in 2003, the water level of the Yangtze River in Three Gorges area rose more than 100 meters. The impact of the man-made reservoir caused by the dam on the surroundings is becoming the object of several studies. In this paper we make use of two long term D-InSAR techniques, the quasi-permanent scatterer technique (QPS) and the Stanford Method for Persistent Scatterer (StaMPS), to measure the deformation trends in Badong, Three Gorges area, China. The results obtained by the two processing tools with the same focused and co-registered data sets are analyzed and compared. Two subsidence areas are identified by both techniques. However, since the QPS analysis is able to process partially coherent targets, many more points are extracted than in StaMPS, and more information can be retrieved.

A new approach is presented in this paper to produce Digital Elevation Model (DEM) in mountainous areas with steep slope using ERS-1/2 tandem data. In order to reduce the impact of phase errors on the Interferometric Synthetic Aperture Radar (InSAR)-generated DEM, an external DEM such as that from Shuttle Radar Topography Mission (SRTM) is utilized in this approach. The proposed algorithm includes two steps: The first step is to model and remove phase trends with a linear regression analysis before converting phase to height; the second step is to filter unreliable height points before interpolating the DEM from the InSAR height map. The critical points are the following: 1) determining the one-to-one correspondence between the interferogram and the SRTM DEM before knowing the InSAR-derived elevation values and 2) estimating the elevation range of every pixel from SRTM DEM. To solve the first problem, an iteratively geocoding algorithm is performed. A DEM interpolation error model solves the second one. For InSAR data processing, the SRTM DEM is not only usable for modeling systematic phase errors but also for filtering gross height errors. The experiments in Zhangbei and the Three Gorges areas in China show that our approach has improved the accuracy of the resulting DEMs significantly without any ground control points.