Urban green spaces, crucial for ecological balance, face global degradation from natural disasters and rapid urbanization. Manual deforestation monitoring is laborious, prompting a shift to remote sensing and bitemporal satellite imagery. Traditional change detection (CD) methods have limitations, but deep learning, especially in semantic CD, shows promise. This study addresses challenges in semantic CD techniques, advocating for comprehensive training on datasets covering both semantic change masks and binary change masks. We propose a novel semantic CD network for urban changes while additionally providing urban greenery increased and decreased regions, integrating deep bitemporal features with an encoder-decoder structure, Atrous spatial pyramid pooling, and a spatial attention module with parallel dilated convolutions. Quantitative assessment, especially with pre-trained VGG16 as a backbone and parallel convolutional layers, demonstrates the proposed method's superiority, showcasing substantial improvements in urban greenery CD alongside overall urban changes. The proposed method holds potential for monitoring climate change, rapid urbanization, and the impact of natural disasters on urban environments, particularly urban greenery.

Korean multi-purpose satellite 3A (KOMPSAT-3A) can acquire electric optical (EO) and mid-wave infrared (MIR) images. Since MIR and EO images provide different information, they can be used together to effectively observe various phenomena on the Earth's surface. However, geometric misalignments exist between the EO and MIR images as the difference in the positions of each sensor when acquiring the images. In this study, we propose a hybrid pyramid matching (HPM) method to conduct the image registration between heterogeneous EO and MIR images with different spatial and spectral characteristics. The HPM method extracts reliable tie points (TPs) by iteratively adjusting the location of local templates in pyramid image pairs. Then, the image registration is conducted using transformation matrix estimated based on the TPs. The HPM method achieved superior accuracy and performance at three different sites.

Cloud cover is a significant obstacle to use optical satellite imagery. Therefore, various studies have been proposed to accurately detect clouds and evaluate satellite image quality. In particular, with the advancement of deep learning technology, many cloud detection studies are being conducted. However, a large volume of high-quality data is required to develop an effective deep learning model training. Thus, in this study, we compare the performance of deep learning cloud detection models for according to the diversity of sensors and resolutions of training data. For conducting the study, five case dataset combinations were constructed and trained with HRNet (High-Resolution Network). The performance evaluation of the trained models was conducted using test images from the KOMPSAT and PlanetScope satellites. As a mean of achieving high cloud detection results, it was found that selecting and using high-quality data is more effective than simply increasing the number of training data.

Image registration is the most basic preprocessing method used to unify coordinates among multitemporal very-high-resolution (VHR) satellite images, thus allowing the acquisition of reliable data of the Earth’s surface. Although image registration requires multiple matching points (MP), false matching points (FMP) are included because of the similar spectral patterns and noise. However, removing FMPs from VHR satellite image pairs is challenging, especially when the images are directly affected by complex factors, such as shadow, relief displacement, and terrain shielding. Therefore, we propose a false matching point removal network (FMPR-net) based on deep learning to eliminate effectively the FMPs to improve registration accuracy. The training dataset is produced by a semi-automatic method. It involves the generation of image patch pairs based on a matching process of scale-invariant feature transform and the assignment of labels referring to the characteristics of true matching points (TMP) and FMPs. The FMPR-net is designed in a Siamese format consisting of two matching point deep feature extractors (MDFE). The architecture of the MDFE consists of one main network and three branch networks to achieve robust extraction of meaningful deep features describing the characteristics of MPs. The FMPR-net removes the FMPs using a true matching probability calculated based on the similarity between deep features. Experiments conducted on four pairs of VHR satellite images have demonstrated that the FMPR-net can effectively remove the FMPs. Consequently, accurate VHR satellite image registration is possible by reducing uncertainty caused by the FMPs.

For the vibration measurement of a structure, contact-type sensors such as gap sensors are used, however, these sensors can add mass-loading to lightweight structures which can result in negative performance. Furthermore, vibrations in a structure caused by multiple factors need to be detected separately to recognize these factors one by one. To solve these problems, in this study, we introduced a target-less photogrammetric vibration measurement technique that measures the vibration by using subpixel-based edge detection and tracking method. Moreover, this technique separates the vibrations occurring in a structure caused by multiple factors by utilizing the phase-based motion magnification technique together with subpixel-based vibration measurement method applied to a video of vibrating structure. The results generated by the proposed method can be used to identify multiple deformations in a structure.

Very-high-resolution (VHR) satellite image contains reliable various information over large areas, so that, it has been used as key data in the field of remote sensing. Image registration must be conducted to effectively use the multitemporal VHR satellite images. Conjugate points (CPs) extracted from the same region between images are required to perform image registration. However, outliers included in the CPs cause distortion when they were used for the image registration. Here we propose a deep learning-based technique to effectively remove the outliers. A Siamese network was built as a purpose of an outlier removal, and the network was trained using data based on the patch pirs centered on each CP. Experimental results demonstrate that the proposed method can remove outliers more effectively than a random sample consensus (RANSAC) technique thus and achieves improved registration accuracy.

Since multi-temporal very-high-resolution (VHR) satellite images generally have geometric misalignment, image co-registration process is required to minimize it. To perform precise image co-registration, extraction of reliable conjugate points (CPs) is an important process. Moreover, CPs extracted from elevated objects can cause severe relief displacements according to acquisition angles of images. In this study, the effect of CPs extracted from buildings on co-registration performance was analyzed. To this end, CPs were extracted using a method that combines feature-based and area-based matching methods, and digital map was used to remove CPs extracted from the buildings. Root mean square errors (RMSE) was calculated using manually obtained checkpoints to evaluate the accuracy of co-registration according to the presence or absence of CPs extracted on buildings. When CPs extracted from buildings were removed, the RMSE of the checkpoints extracted from the dense-building area was improved by more than 4 pixels.

Fine co-registration that precisely aligns multiple images acquired over a given area is an important process to exploit the very high resolution (VHR) multitemporal images in a wide range of remote sensing applications. The objective of this study is to analyze the effect of the fine co-registration performance on an unsupervised change detection between VHR images. To this end, we extract registration noise (RN) samples, which are denoted as misaligned pixels in a local region. Then, the location of conjugate points (CPs) is positioned by analyzing the local distribution of the extracted RN samples. The CPs are employed for generating a non-rigid transformation model to warp a sensed image into a reference image. An unsupervised change vector analysis approach is used to validate the effectiveness of the proposed fine co-registration performance. Experiments are implemented on a Worldview-3 VHR multispectral dataset.

Unmanned aerial vehicle (UAV) systems are becoming increasingly important since fine spatial and high temporal resolution data have been unobtainable from traditional remote sensing platforms. Advanced UAV data can provide a great opportunity for disaster monitoring, including building damage assessment. Hurricane Harvey struck southern Texas, U.S.A, on August 26, 2017, causing catastrophic flooding and storm damage. We visited Holiday Beach, which suffered severe building damage, and conducted UAV surveying for building damage assessment. In this study, a region growing scheme was proposed to estimate the building damage, considering building elevation change and the spectral difference of debris. In addition, different weight coefficients of a digital elevation model (DEM) were investigated. The results showed that the proposed method can be used for high-definition building damage assessment through accurate detection of the intact building regions.

Very High Resolution (VHR) multitemporal images that have been already applied registration process still show a residual misalignment due to the dissimilarities of the acquisition environment. The objective of this paper is to mitigate the residual misalignment between VHR images to get a fine geometric alignment result. Here we propose to refine the local misalignment between VHR images by extracting Registration Noise (RN), which is denoted as misaligned samples. Extracted RN pixels are used as Control Points (CPs), and local distribution analysis of the CPs in a specific region defined by a quadtree structure is carried out for finding their correspondences. Matched CP pairs are employed for generating a deformation map to warp the sensed image to the master image. Experiments carried out on both simulated and real multitemporal VHR datasets acquired from IKONOS and QuickBird sensors confirm the validity of the analysis of the proposed method.

In this paper, a segmentation-based approach to fine registration of multispectral and multitemporal very high resolution (VHR) images is proposed. The proposed approach aims at estimating and correcting the residual local misalignment [also referred to as registration noise (RN)] that often affects multitemporal VHR images even after standard registration. The method extracts automatically a set of object representative points associated with regions with homogeneous spectral properties (i.e., objects in the scene). Such points result to be distributed all over the considered scene and account for the high spatial correlation of pixels in VHR images. Then, it estimates the amount and direction of residual local misalignment for each object representative point by exploiting residual local misalignment properties in a multiple displacement analysis framework. To this end, a multiscale differential analysis of the multispectral difference image is employed for modeling the statistical distribution of pixels affected by residual misalignment (i.e., RN pixels) and detect them. The RN is used to perform a segmentation-based fine registration based on both temporal and spatial correlation. Accordingly, the method is particularly suitable to be used for images with a large number of border regions like VHR images of urban scenes. Experimental results obtained on both simulated and real multitemporal VHR images confirm the effectiveness of the proposed method.

Even after coregistration, very high resolution (VHR) multitemporal images acquired by different multispectral sensors (e.g., QuickBird and WordView) show a residual misregistration due to dissimilarities in acquisition conditions and in sensor properties. Residual misregistration can be considered as a source of noise and is referred to as registration noise (RN). Since RN is likely to have a negative impact on multitemporal information extraction, detecting and reducing it can increase multitemporal image processing accuracy. In this letter, we propose an approach to identify RN between VHR multitemporal and multisensor images. Under the assumption that dominant RN mainly exists along boundaries of objects, we propose to use edge information in high frequency regions to estimate it. This choice makes RN detection less dependent on radiometric differences and thus more effective in VHR multisensor image processing. In order to validate the effectiveness of the proposed approach, multitemporal multisensor data sets are built including QuickBird and WorldView VHR images. Both qualitative and quantitative assessments demonstrate the effectiveness of the proposed RN identification approach compared to the state-of-the-art one.

We propose a modified IR-MAD based on the generation of synthetically fused images in order to minimize the effect of change detection results corresponding to noise and feature reduction. Synthetically fused hyperspectral images were first generated using a cross-sharpening algorithm. MAD variates according to each pair of synthetically fused images were then calculated to reduce the influence of data noise in the hyperspectral image. In particular, we applied the integration of MAD variates in this study. To evaluate the performance of our algorithm, we constructed a hyperspectral dataset using the Hyperion sensor and analyzed the data noise and bands of principal components.

Very High Resolution (VHR) multitemporal images show a residual misalignment even after applying effective state of the art co-registration. This residual misalignment is caused by the dissimilarities of the acquisition circumstances such as off-nadir angle of the sensor, stability of the acquisition platform, structure of the considered scene, and so on. This paper aims at mitigating the residual misalignment of VHR multitemporal images to get a fine co-registration result. Here we propose to use Registration Noise (RN), which represents misaligned samples, for refining co-registration. After standard co-registration, a local analysis of RN pixels is fulfilled for extracting Control Points (CPs) and matching them according to the amount of the RN pixels. Matched CPs are employed for generating a deformation map to warp one image to the other image. Experiments carried out on both simulated and real multitemporal VHR images acquired by QuickBird sensors confirm the validity of the analysis and effectiveness of the proposed method.

Even after applying effective coregistration methods, multitemporal images are likely to show a residual misalignment, which is referred to as registration noise (RN). This is because coregistration methods from the literature cannot fully handle the local dissimilarities induced by differences in the acquisition conditions (e.g., the stability of the acquisition platform, the off-nadir angle of the sensor, the structure of the considered scene, etc.). This paper addresses the problem of reducing such a residual misalignment by proposing a fine automatic coregistration approach for very high resolution (VHR) multispectral images. The proposed method takes advantage of the properties of the residual misalignment itself. To this end, RN is first extracted in the change vector analysis (CVA) polar domain according to the behaviors of the specific multitemporal images considered. Then, a local analysis of RN pixels (i.e., those showing residual misalignment) is conducted for automatically extracting control points (CPs) and matching them according to their estimated displacement. Matched CPs are used for generating a deformation map by interpolation. Finally, one VHR image is warped to the coordinates of the other through a deformation map. Experiments carried out on simulated and real multitemporal VHR images confirm the effectiveness of the proposed approach.

The objective of this paper is to extract a suitable number of evenly distributed matched points, given the characteristics of the site and the sensors involved. The intent is to increase the accuracy of automatic image-to-image registration for high-resolution multisensor data. The initial set of matching points is extracted using a scale-invariant feature transform (SIFT)-based method, which is further used to evaluate the initial geometric relationship between the features of the reference and sensed images. The precise matching points are extracted considering location differences and local properties of features. The values of the parameters used in the precise matching are optimized using an objective function that considers both the distribution of the matching points and the reliability of the transformation model. In case studies, the proposed algorithm extracts an appropriate number of well-distributed matching points and achieves a higher correct-match rate than the SIFT method. The registration results for all sensors are acceptably accurate, with a root-mean-square error of less than 1.5 m.

Precise image-to-image registration is required to use multi-sensor data implementing a diversity of applications related with remote sensing. The purpose of this paper is to develop an automatic algorithm that co-registers high-resolution optical and SAR images based on an integrated intensity-and feature-based approach. As a pre-registration step, initial differences between the translation of the x and y directions between images were estimated with the Simulated Annealing optimization method using Mutual Information as an objective function. After the pre-registration, the line features were extracted to design a cost function that finds matching features based on the similarities of their locations and gradient orientations. Only one feature at each regular grid region having a minimum value of cost function was selected as a final matching point to extract the large number of well-distributed points. The final points were then used to construct a transformation combining the piecewise linear function with the affine transformation to increase the accuracy of the geometric correction.

It is generally difficult to classify an object type having different colors into the same class using only optical data such as a satellite or aerial image. This paper proposes a method that solves this problem by combining LiDAR data and an aerial image. The method extracts building pixels from LiDAR data and then identifies building objects on the aerial image by overlaying the LiDAR result to a segmented aerial image through the definite rule. This process plays a role in transforming building objects of LiDAR data to ones of the aerial image.