In this letter, we propose a strategy for unmanned aerial vehicle (UAV) image stitching to generate natural-looking panoramas. Traditional methods using homography to perform alignment cannot account for images with parallax, so they require that the input images should be taken from the same viewpoint or the scene should be near the planar. However, remote sensing images obtained by UAVs usually do not satisfy such an ideal situation, and the stitching results always suffer from artifacts. To overcome these challenges and obtain natural-looking panoramas, a global alignment strategy is proposed to better align the input images. Combined with a shape-preserving warp, the stitching results can achieve better alignment accuracy while maintaining the shape. Meanwhile, locality preserving matching (LPM) is used to eliminate mismatches during feature detection and matching for accurate alignment. In addition, to make the stitching results more natural-looking, we also use multiband blending to eliminate artifacts that may exist in the results due to unmodeled effects. Experiments show that our stitching strategy can effectively improve alignment accuracy and obtain natural-looking results compared to other state-of-the-art methods.

Building footprint extraction plays an important role in many remote-sensing (RS) applications such as urban planning and disaster monitoring. Mainly, the exploitation of contextual information in a fixed receptive field is the focus of previous research, which makes it difficult to generically extract buildings that vary greatly in size and shape, especially when isolated large buildings are surrounded by dense small buildings. To improve this problem, we attempt to teach the network to adjust the receptive field and enhance useful feature information adaptively. In this article, we propose a novel adaptive screening feature network (ASF-Net), which can independently screen and enhance effective feature information from two aspects. On the one hand, we propose a deepened space up-sampling block to screen useful information and help establish boundaries. On the other hand, we propose an Adaptive Information Utilization Block (AIUB) to enlarge the receptive field of feature maps and refine the incomplete building footprint. As a result, the more accurate multiscale building footprint is inferred from the enhanced features. Experimental results on the popular aerial image segmentation datasets show that ASF-Net obtains competitive results [80.2% intersection over union (IoU) on the Inria aerial image labeling dataset and 74.2% IoU on the Massachusetts buildings dataset] in comparison with several state-of-the-art models. The TensorFlow implementation is available athttps://github.com/jyx0516/ASF-Net.

Feature matching is the foundation and key task of remote sensing image registration, which is to establish a reliable point corresponding relationship between the feature points of two images. In this article, a simple and effective local consensus method for rigid and nonrigid feature matching is proposed and applied to solve the problem of high outliers ratio caused by nonrigid transformation, nonlinear radiation difference, and speckle noise in the remote sensing image registration task. We first establish the putative feature correspondences according to the similarity between local descriptors and then use local consensus constraints (including neighborhood consensus and motion vector consensus) to remove outliers. The specific steps are given as follows. First, we use the neighborhood consensus constraint of feature points to carry out preliminary filtering to remove outliers with obvious errors and retain a large number of inliers, so as to obtain a clean reliable set. Then, the reliable set space is grided into several nonoverlapping cells, and the estimated motion vector is calculated for each cell. By taking the comprehensive deviation between the ordinary motion vectors and estimated motion vectors, we transform the matching problem into a mathematical optimization model and derive a closed-form solution with linear time and linear space complexities. In this way, our method can also significantly increase the speed of operation without sacrificing accuracy. A large number of feature matching experiments on remote sensing prove that our method is superior to existing methods and also has good results in the general scene.

Multi-focus image fusion technology is to extract different focused regions of the same scene among partially focused images and merge them together to generate a composite image where all objects are clear. Two crucial points to multi-focus image fusion are the effective focus measurement method to evaluate the sharpness of the source images and the accurate segmentation method to extract the focused regions. In conventional multi-focus image fusion methods, the decision map obtained according to the focus measurement is sensitive to mis-registration, or produces an uneven boundary lines. In this paper, the maximum value in the top-hat transform and the bottom-hat transform is used as the gradient measurement value, and the complementary features between multiple scales are used to achieve accurate focus measurement for initial segmentation. In order to obtain a better fusion decision map, a robust image matting algorithm is used to refine the trimap generated by the initial segmentation. Then, make full use of the strong correlation between the source images to optimize the edge regions of the decision map to improve the image fusion quality. Finally, a fusion image is constructed based on the fusion decision map and the source images. We perform qualitative and quantitative experiments on publicly available databases to verify the effectiveness of the method. The results show that compared with several state-of-the-art algorithms, the proposed fusion method can obtain accurate decision maps and achieve better performance in visual perception and quantitative analysis.

In this paper, we propose a novel and practical convolutional neural network method for building footprint generation in remote sensing images, in order to deal with the problem that the detailed information and geometric structure of ground objects in high-resolution images become more abundant, which leads to a large increase in the calculation amount. So we introduce a deepened space module, which can ignore the channels with weak target features and emphasize the effective features. It is embedded in each splicing layer in the upsampling process of U-net to achieve the effect of feature selection. By means of clipping and data enhancement, we carry out iterative training and model optimization learning on Inria aerial image label dataset, and realize the automatic generation of building footprint. Compared with FCN8s, Unet, SegNet, PSPNet, Deeplabv3 + and GLNet, experimental results show that the method we use to generate building footprint is more accurate, and in IoU, mPA, PA three indicators are better than the comparison algorithms.

In view of the difficulty and low accuracy of small object detection in remote sensing images, this paper proposes a bidirectional cross-scale connection feature fusion network with an information direct connection layer and a shallow information fusion layer. Aiming at the problem that the detection targets in remote sensing images are mainly small and medium-sized targets, we fuse the shallow feature maps with rich spatial information in the bidirectional cross-scale connection feature fusion network instead of directly using the shallow feature maps for regression and classification. While ensuring the model inference speed, the detection accuracy of small objects is improved. At the same time, we use the information direct connection layer to perform feature fusion with the initial information in each iteration of the bidirectional cross-scale connection feature fusion pyramid to prevent the loss of small object information. Experimental results show that the algorithm proposed in this paper can obtain good accuracy and real-time performance on the NWPU VHR-10 dataset.

In rural areas where disasters occur frequently, the calculation of building areas is crucial in property assessment. In the segmentation algorithm, Mask R-CNN can distinguish the adjacent objects and extract the outline of an object. Based on this observation, we propose a novel method to calculate the building areas based on Mask R-CNN and adopt the concept of transfer learning to train our model, which can achieve good results with a small number of drone aerial images as training samples. The proposed method involves three main steps: 1) pretraining using open-source satellite remote sensing images; 2) fine-tuning with a small number of drone aerial images; and 3) testing with new images and area calculation based on the number of building pixels. The experiments show that the proposed method can achieve good results in terms of F1 score and intersection over union.

This article proposes a strategy for drone image stitching using local mesh-based bundle adjustment and shape-preserving transform, which aims to effectively stitch multiple overlapping drone images into a natural panoramic image. Existing traditional methods using a simple homography cannot handle the situation that the input drone images have parallax effect, and the image mosaic result always suffers from artifacts. In order to achieve natural-looking stitching results without the above limitation, we divide the proposed method into the following steps. Starting from initial feature sets obtained by off-the-shelf feature extraction methods, we incorporate the parallax errors into an energy minimum framework and construct a robust alignment energy. This energy can be minimized efficiently based on local bundle adjustment and robust$3\sigma $principle, which could eliminate parallax effects and achieve accurate alignment. Then the seamless panoramic image is obtained by warping the target image and the source images onto the mesh plane directly. An image patch can be transformed by projective transformation (e.g., homography), which provides good alignment but may cause distortions. Consequently, combined with mesh-based shape-preserving transform, our proposed strategy can improve the naturalness of the results flexibly. Experiments show that our stitching strategy can eliminate parallax effects more effectively and achieve natural-looking results compared to other state-of-the-art methods.

In this paper, in order to enhance the infrared target in infrared image and retain the edge and detail information in visible image, we propose a multi-scale decomposition fusion method based on phase congruency and saliency. In this method, the Laplacian pyramid is first used to decompose the source image into detail layers and base layers. Secondly, we use a method based on phase congruency for the fusion of detail layers. Thirdly, for the base layer, we decompose it into saliency map and residual map. The “max absolute” rule and “averag” rule are adopted for the fusion of saliency map and residual map, then the fused saliency map and residual map are added to attain the fused base image. Finally, we use the inverse transform of Laplacian pyramid to reconstruct the fused image. The experimental results show that the proposed method have better fusion effect than other methods. What's outstanding is that the infrared targets in the fused image are enhanced and abundant edges are preserved.

This paper proposes a strategy for drone image stitching using local least square alignment, which aims to effectively stitch multiple overlapping drone images into a natural panoramic image. Existing traditional methods using simple homography cannot handle the situation that the input drone images have parallax effect, and the mosaic result always suffers from artifacts. In order to achieve natural-looking stitching results without the above limitation, we divide the proposed method into the following two steps, namely, local least square alignment and global similarity constraint. Starting from initial feature sets obtained by traditional feature extraction methods, we construct a robust alignment energy based on parallax errors to adaptively eliminate parallax effects. The energy can be efficiently minimized used least square estimate. Combined with global similarity constraint, our proposed strategy can flexibly improve the naturalness of the results. Experiments show that our stitching strategy can more effectively eliminate parallax effects and achieve natural-looking results compared to other state-of-the-art methods.

The goal of unmanned aerial vehicle (UAV) image mosaicing is to create natural-looking mosaics free of artifacts due to the parallax of the image and relative camera motion. UAV remote sensing is a low-altitude technology and the UAV imaged scene is not effectively planar, yielding parallax on images. In this paper, we apply local homography to match UAV images, which can reduce misalignment artifacts or “ghosting” in the results compared with 2D projective transforms or global homography. In addition, when an object in three dimensions is mapped to an image plane, different surfaces have different projections. These projections vary with the viewpoint in a sequence of UAV images, which still causes artifacts near some tall buildings if we only use local homography. We propose a novel stitching method based multi-region local projection deformation, that divides the overlapping regions of input images into several regions, then meshes image to calculate local projections by partitioned regions. Specifically, we use a strategy where multiple regions have different weights for calculating local projections, which can significantly reduce ghosting due to these projections vary with the viewpoint and parallax. The benefits of the proposed approach are demonstrated using a variety of challenging cases.

Focusing on the characteristics of remote sensing images, this study proposes a new algorithm for feature matching of remote sensing images to eliminate mismatch. The algorithm utilizes feature descriptors, such as scale-invariant feature transform, for rough correspondence and the thin-plate spline for non-rigid transformation. Under the Bayesian framework, correspondence and transformation are alternately optimized by the expectation-maximization algorithm to automatically eliminate mismatched points. We also introduce a local geometrical constraint to maintain the internal structure of adjacent feature points. We apply this method to a large number of remote sensing images, and the experimental results reveal the method's superiority over the state-of-the-art.

Image stitching stitches multiple overlapping images into a seamless image according to the corresponding geometric relationship between the reference and source images. In this study, the parallax-tolerant image stitching method based on robust elastic warping is applied to the stitching of drone images, and locality-preserving feature matching is used to effectively remove outliers from the drone images. The method can be divided into three stages, namely, locality-preserving feature matching, robust elastic warping, and global projectivity preservation. First, a set of high- precision point matching is provided for a drone image, and local matching is used. Second, the robust elastic warping function eliminates the parallax error, and the input image is distorted according to the calculated deformation on the grid plane. Finally, the global projectivity-preserving method is applied to obtain high-precision result panoramas. Experiments on several sets of drone images demonstrate that our method can generate better panoramas over the competitors.

This study introduces a robust method for panoramic unmanned aerial vehicle (UAV) image mosaic. The traditional automatic panoramic image stitching method requires the camera to carefully rotate the optical center to obtain an image, but the image used for mosaic in reality cannot easily achieve this ideal state. In particular, remote sensing images obtained by UAVs do not satisfy such a situation. The images may not be on a plane yet, and several of them may even have non-rigid changes. Therefore, the classical method of UAV image stitching is expected to produce poor results. To this end, we improve the traditional stitching method to overcome the abovementioned challenges. Specifically, a non-rigid matching algorithm is introduced to the system to make it suitable for remote sensing images. We perform bundle adjustments using a new strategy to make the mosaic system suitable for UAV images. Experimental results show that our method is more robust than the traditional method.