Military conflicts have a significant impact on national security and the ecological environment. Effective detection methods for ground-moving objects in complex scenarios can be further utilized to assess damage and provide recommendations for security and environmental restoration. Current ground-moving object detection in optical remote sensing imagery struggles with balancing detection accuracy and real-time performance, hindering timely threat assessment. To address this, the study proposes ground-moving object detector (GMODet), a real-time detection method incorporating region awareness and semantic-spatial interaction to enhance the detection of partially occluded and fine-grained objects in complex environments. The framework includes three modules: the region awareness module (RAW), cross-scale context-aware feature aggregator (CCFA), and semantic-spatial progressive interaction module (SPIM), focusing on extracting discriminative features for contextual, multiscale, and semantic-spatial information. A new dataset, ground-based moving object dataset (GMOD), is constructed with four object types and high scene complexity, alongside experiments on the publicly available military vehicle remote sensing dataset (MVRSD). GMODet achieves the state-of-the-art performance, with mAP50, mAP75, and mAP scores of 65.5%, 48.5%, and 42.3% on the GMOD, outperforming the second-best results by 1.9%, 5.1%, and 1.5%, respectively. On the MVRSD, it achieves mAP50, mAP75, and mAP scores of 88.2%, 75.2%, and 61.7%, respectively. Notably, with an inference time of just 25 s on large-scale images ($9152\times 9152$pixels), GMODet showcases outstanding accuracy, speed, robustness, and generalization in ground-moving object detection.

In order to improve the accuracy of localization of projectile impact point, detection and localization methods of projectile impact point based on remote sensing imagery are studied in this paper. We constructed a labeled dataset named SFP about smoke, fire, and impact point containing 6700 images. We designed a two-stage segmentation algorithm based on prior knowledge. Our algorithm first segments the region containing smoke and fire, which is treated as the Region of Interest (ROI). Within the ROI, our algorithm then segments impact point. Our algorithm is compared with the direct segmentation algorithm on SFP. The precision, recall, and mAP of our algorithm are 0.693, 0.654, and 0.639. And those of the direct segmentation algorithm are 0.263, 0.175, and 0.041. The performance of our algorithm is much higher, which indicates that our algorithm can accomplish the task of detecting and localizing the impact points based on remote sensing imagery better.

Satellite video multi-object tracking holds significant importance in national defense security, emergency rescue, urban planning, and various other applications. This task is particularly challenging due to the complex background of remote sensing images and the dynamic nature of spatiotemporal processes. Currently, satellite video object tracking faces issues like erroneous detection and tracking, especially in cases of low contrast and missed detection and tracking for small objects. To address these challenges, this study introduces a method for multi-object tracking in satellite videos that considers both global information and local features of objects. The proposed method also incorporates weak feature enhancement techniques, specifically targeting issues such as low contrast. Extensive experimental verification on Jilin-1 satellite video data has been conducted, yielding excellent results and demonstrating the effectiveness of the proposed approach.

Eye-controlled human-computer interaction (ECHCI) attracts attention for its human-centered, natural and direct operation characteristics. The most common ECHCI trigger motion is “gazing,” which causes Midas Touch problems, lowering usability and operation experience. This paper innovatively proposed using motion combinations based on blinking as the trigger of interactive objects (IOs). Trigger motions and IO design parameters were explored on the platform consisting of Tobii eye-tracker and computer vision kits. In the motion experiment, it was concluded 2-blinks could bring a low task load and high success rate of 95%. In the IO size experiment, the larger the IO diameter, the shorter the users’ response time and the higher users’ interaction success rate, and 55.5 mm was the optimal setting. Based on this, it was found a larger spacing could bring higher operation efficiency, and 33.3 mm was taken as the recommended value. Based on the conclusion above, an eye-controlled multimedia player prototype was built for usability evaluation, which showed an equally high efficiency compared to mouse-controlled HCI. The findings in this paper overcame Midas Touch issues and avoided the interference of unavoidable involuntary blinking, which brought a significant increase in system robustness and has a wide range of potential applications.

High spatial, high spectral, and high temporal ($\text {H}^{3}$) information of the objects of interest can be provided by hyperspectral video, which makes it possible to track objects in complex scenarios. However, during motion, changes in the target’s appearance, background, and spectral information can degrade the performance of existing hyperspectral trackers due to insufficient training data. Consequently, this results in weak generalization of these trackers. In this article, to solve the above problems, a deep temporal-spectral–spatial anchor-free Siamese tracking network for hyperspectral video object tracking, namely HA-Net, is proposed. In HA-Net, a Siamese spectral enhancement tracker module based on an RGB tracker (pseudo-color tracker) is designed, which uses the powerful feature expression capabilities of the deep network to learn more discriminative deep spectral features for identifying objects in complex scenarios. The pseudo-color tracker is introduced to solve the problem of model performance limitation due to insufficient training data. By introducing the temporal-spectral–spatial online discrimination learning module, the temporal-spectral–spatial information of the target can be dynamically modeled to adapt to new targets and the dynamic changes of targets. Benefiting from the double Siamese network architecture, the model can be effectively trained from scratch with less than 20 000 training samples. Online learning of temporal-spectral–spatial information for the target, particularly in cases of insufficient training data, can alleviate the issue of model degradation. This approach enhances the model’s robustness when tracking the target in complex scenes. In the 2021 IEEE WHISPERS Hyperspectral Object Tracking (HOT) Challenge, HA-Net obtained the best performance, with a distance precision (DP) score of 0.948 and an area under the curve (AUC) score of 0.688. The running speed is also 14 frames/s, which is superior to the existing hyperspectral object trackers for hyperspectral video. The source code is available athttps://github.com/zhenliuzhenqi/HOT.

Matching multimodal remote sensing images (RSIs) remains an ongoing challenge due to the significant nonlinear radiometric differences and geometric distortions, resulting in matches exhibiting one-to-many matches or mismatches. To tackle this challenge, we propose a novel approach for multimodal RSI matching called modality-independent consistency matching (MICM), which leverages the capabilities of deep convolutional neural networks and the transformer attention mechanism to improve the matching performance. The proposed MICM method consists of three key steps. First, a Unet-like feature extraction backbone network is employed to learn multiscale invariant features from multimodal RSIs, enabling the extraction of rich and evenly distributed feature keypoints. Second, a hybrid approach combining local learning features with the transformer attention mechanism is introduced to aggregate learning features, facilitating both detailed capture and long-range modeling to enhance the representation ability of the features. Third, a feature consistency correlation strategy is adopted to maximize the number of correct corresponding feature points, ensuring reliable matching performance. The performance of the proposed method has been extensively evaluated on both the same scene and different scene multimodal RSIs, which are captured from various imaging modes, wavebands, and platforms. The results show the superior matching performance of the proposed MICM method compared to commonly used and state-of-the-art handcrafted- and learning-based methods when evaluated on both the same scene and different scene datasets. The proposed method serves as a valuable reference for addressing common challenges in multimodal RSI matching.

Existing change detection (CD) research mostly focuses on pixel-level dense prediction, while object detection (OD) for entire damaged/changed buildings is rare. Object-level detection datasets for damaged/changed buildings are also lacking. This article proposes an object-oriented damaged/changed building CD model, OoCDNet, along with five global-scale OD datasets for damaged/changed buildings. OoCDNet bridges and integrates the dual tasks of CD and OD, driven by the task of locating damaged/changed buildings. By modeling the changes in buildings at the object level between bitemporal images, it achieves rapid identification of target buildings. OoCDNet consists of four parts, the two paths of the dual-path feature extraction module (DouBackbone) are responsible for extracting the base features of the image, the bidirectional pyramid feature aggregation module (AggNeck) models the change information of the features aggregated at the end, the cross-informative self-attentive short-circuit enhancement module enhances the features from the DouBackbone with highly efficient self-attentive information and supplements the enhancement information to the AggNeck, and the enhanced features with semantic and localization information are fed into the detection module for OD. The proposed OoEWEBD is a global-scale OD dataset for damaged buildings, containing 10377 image pairs, each sized$256\times 256$pixels. The remaining four datasets are created based on the WHU_CD, LEVIR-CD+, S2Looking, and xBD datasets, targeting buildings with general changes or those affected by disasters. Compared with state-of-the-art (SOTA) OD and CD methods, OoCDNet can quickly and effectively detect target buildings, achieving the highest accuracy and having a high application value. The code and datasets will be available athttps://github.com/Haiming-Z/OD-based-Change-Detection-mode-OoCDNet.

Multimodal remote sensing image (MRSI) high-precision matching faces significant challenges due to the large nonlinear distortions in radiation and geometry. To address this problem, we propose a novel MRSI matching method, multimodal remote sensing image matching with Delaunay triangulation constraint network (M2DT-Net), which takes advantage of deep convolutional neural networks (CNNs) and the Delaunay triangulation (DT) strategy. First, a fine-tuned CNN model is applied to learn invariant features with high robustness across MRSIs. Second, an adaptive feature descriptor distance constraint combined with the DEGENSAC refinement is employed to filter the obtained matches, and only the matches with high credibility are retained. Finally, a DT strategy is adopted to expand more underlying correct matches that are dropped inliers in the previous stage. The experimental results on seven types of MRSI pairs indicate that M2DT-Net is superior to seven state-of-the-art image matching algorithms in terms of model performance and efficiency, achieving a balance of robustness and time consumption. The image registration results further demonstrate the effectiveness of the proposed algorithm. Therefore, the proposed M2DT-Net method provides a reference for resolving common multimodal image matching problems.

Due to huge differences in radiation characteristics and geometric characteristics of multi-source remote sensing images, presenting a big challenge for high-precision registration. In this paper, a new registration method based on deep learning is proposed. First, we use the convolutional neural network to extract deep learning features of the reference and sensed image after adaptive down-sampling, and extract 512-dimensional descriptor on the feature map to calculate the matching result, homography matrix and overlap area. Then, the circumscribed rectangle of the overlapping area is divided into blocks, and the same name point information extracted from all sub-blocks is combined to obtain matching result of source image pair, and then homography matrix of the source image pair is estimated. Finally, the registration result is obtained. Results show that the proposed algorithm has strong adaptability and robustness in the registration of multiple heterogeneous images in mountains, hills and plains.

The vectorization of buildings provides quantitative disaster information and building damage extraction for the accurate assessment of disasters. This study presents an automatic extraction technology for flat-roofed houses by using high-resolution remote sensing images based on the single-point extraction of such houses and the concept of stepwise image segmentation. Vegetation removal, multi-scale segmentation, k-means clustering segmentation, rectangle recognition, and morphological processing are adopted to identify the center of an assumed house in the image by considering the characteristics of remote sensing images and the main features of house recognition (i.e., spectral characteristics and shape features). Furthermore, the single-point extraction of flat-roofed houses is used to acquire the surface vector diagram of the extracted houses, and finally, achieve the automatic extraction of flat-roofed houses. Experimental result shows that the accuracy of the vectorization extraction of flat-roofed houses through the combination is 83.33%.

As a fundamental problem, spatial resection based on collinearity equations has been a subject of study since the beginning of photogrammetry. Owing to the nonlinearity of collinearity equations, the standard Newton-Raphson method requires approximate values for unknown parameters. Therefore, the method is inapplicable to modern oblique photogrammetry. In this letter, we propose a new altitude spatial resection algorithm that combines closed-form solutions and iterative solutions. In this algorithm, a set of homotopy algorithm solutions is first selected as the initial value on the basis of the minimum sum of the squared errors. Quasi-Newton method is then used for nonlinear iteration. Experimental results obtained from different data sets indicate excellent performance compared with serial implement.

Inclined imaging is an advanced sensing technique, and has been extensively used. Therefore, large-rotation angle photogrammetric resection has become an important topic. However, traditional iterative methods are limited by requiring good initial values. By contrast, noniterative methods do not require an initial value, although they exhibit relatively low accuracy and robustness. To obtain results with superior precision and universality, this letter proposes an improved approach by modifying the initial value acquisition and iterative methods. This algorithm uses nonlinear iteration to reduce the model error, thereby possibly achieving an exceptional convergence for large-rotation-angle photogrammetric resection. Experimental results on the real data indicate that the proposed algorithm outperforms the previous methods.

Due to repetitive patterns and gray changes in the remote sensing images, feature-point-based registration methods generally fail to determine every correctly matched point. In this letter, a novel registration algorithm that uses point structure information, which includes an improved shape context in feature description and consensus graph emerging from putative matches in feature matching, is proposed. First, to obtain robust initial matching point pairs, a DAISY descriptor is combined with a shape context descriptor that is improved using 1-D Fourier transformation. Second, the final matching results are estimated using graphic transform matching based on the local structure information of the point to remove outliers from initial correspondences. Finally, experimental results demonstrate that the proposed approach, based on structure information, is robust and can improve alignment precision in particularly complex environments.

This paper proposes a novel kernel change detection algorithm (KCD). The input vectors from two images of different times are mapped into a potential much higher dimensional feature space via a nonlinear mapping, which will usually increase the linear margin of change and no-change regions. Then a simple linear distance measure between two high dimensional feature vectors is defined in features space, which corresponds to the complicated nonlinear distance measure in input space. Furthermore the distance measure's dot product is expressed in the combination of kernel functions and large numbers of dot product processed in input space by combined kernel tactic, which avoids the computational load. Finally this paper takes the soft margin single-class support vector machine (SVM) to select the optimal hyper-plane with maximum margin. Preliminary results show the kernel change detection algorithm (KCD) has excellent performance in accuracy.