Detecting vehicles from satellite videos presents several significant challenges: 1) Satellite video frames typically possess extremely high resolutions, often containing millions or even hundreds of millions of pixels, whereas onboard computational resources remain constrained; 2) The non-uniform spatial distribution of vehicles results in inefficient allocation of computational resources; 3) Vehicles are typically small with limited distinguishing features, further complicating the detection task. In this article, we propose LSCNet (Large Scenes to Clusters), an efficient and lightweight satellite video vehicle detection network specifically designed to address these challenges. To alleviate the difficulties associated with large-scale imagery and uneven vehicle distributions, we introduce a plug-and-play Object Cluster Module (OCM). The OCM leverages inter-frame information from satellite video to adaptively identify and prioritize clustered regions, thereby enhancing detection precision. Furthermore, to improve the extraction of discriminative features from small-sized vehicles, we propose a lightweight Multi-frame Feature Aggregation Module (MFAG), which effectively captures the spatiotemporal characteristics of vehicles while maintaining computationally efficiency. Additionally, we refine the regression loss function by integrating the Kullback-Leibler Divergence (KLD), enabling the generation of higher-quality bounding boxes and significantly boosting the detection performance for small objects. Experimental evaluations on the Jilin-1 satellite video dataset demonstrate that the proposed method achieves improved detection accuracy while maintaining real-time performance, thereby validating its robustness and practical effectiveness.

The rapid advancements in video satellite technology have enabled dynamic Earth observation. However, the unique characteristics of satellite video data, such as extensive spatial coverage, sparse motion information, and significant foreground–background imbalance, introduce numerous challenges for practical applications. To address these difficulties, the task of event of interest (EOI) detection has emerged as a critical solution by extracting meaningful regions containing dynamic information from satellite videos. In this article, a novel deep learning framework for EOI detection, integrating spatiotemporal slice analysis and metric learning to tackle these challenges is proposed. The framework employs spatiotemporal slice analysis to convert 3-D motion information into 2-D motion trajectories, preserving essential dynamic information while minimizing background redundancy. A ResNet-based network is adopted for robust feature extraction, and a custom Earth mover’s distance (EMD) metric is introduced to enhance detection precision, enabling accurate differentiation between EOIs and non-EOIs in satellite video data. Experimental results demonstrate the superior performance of the proposed method.

Satellite video scene classification (SVSC) is a critical task for dynamic earth observation. However, it remains challenging due to the distinct spatiotemporal characteristics of satellite videos and the scarcity of annotated data, both of which differ significantly from general video datasets. Although Vision Transformers (ViTs) have shown strong performance in general video classification, their direct application to SVSC often results in suboptimal performance and overfitting. To address these challenges, we propose PESAT, a novel parameter-efficient spatiotemporal adapter tuning framework specifically tailored for SVSC tasks. PESAT enables the effective adaptation of pre-trained ViTs for SVSC by keeping the backbone model largely frozen and fine-tuning only a small number of strategically inserted adapter modules. Our framework incorporates three key innovations: an efficient temporal attention modeling (TAM) mechanism that reuses pre-trained self-attention weights for temporal feature extraction without adding new parameters; a sensitivity-guided adapter insertion strategy that identifies optimal locations within the ViT to place adapters, maximizing their impact; and a hybrid gated adapter (HGA) module, which combines depthwise convolution and a dynamic gating mechanism to capture complex spatiotemporal contexts specific to satellite video data. Experimental results demonstrate the superior performance of the proposed method.

Satellite video single object tracking (SVSOT) stands as a pivotal research area. However, it faces significant challenges in low-light environments, particularly when dealing with dark and weak vehicles. Previous studies have predominantly focused on tracking methods under favorable lighting conditions, neglecting the complexities introduced by inadequate illumination. The difficulty in extracting features from targets in low-light environments, coupled with the susceptibility of dark and weak targets to background noise, exacerbates these challenges. In low-light environments, dark and weak vehicles exhibit less distinctive features and are more susceptible to background interference due to the reduced contrast. To tackle the above challenges, this work proposes an innovative correlation filter (CF)-based tracker (RETrack) that incorporates a retinex-inspired target enhancement. This enhancer integrates an effective low-light enhancement within the CF-based tracker, enhancing target visibility by reallocating target energy based on the characteristics of target motion. Moreover, to mitigate background interference and leverage background information efficiently, an adaptive label update mechanism is developed to suppress background disturbance. Furthermore, this work constructs a satellite video low-light tracking benchmark SATDark, which comprises 120 sequences of dark and weak vehicles. Comprehensive experiments show that RETrack surpasses current leading trackers on the SATDark, providing innovative insights and advancing the field of satellite video object tracking. Additionally, RETrack achieves real-time processing speeds exceeding 30 frames/s on a single CPU, underscoring its practical applicability and efficiency.

Semantic segmentation is a crucial task in remote sensing and has been predominantly performed using convolutional neural networks (CNNs) for the past decade. Recently, transformers with self-attention mechanisms have demonstrated superior performance compared to CNNs. However, due to the locality of CNN and the high computational complexity and massive data resource requirements of transformer, neither of them can be well applied in resource-constrained practical remote sensing scenarios. Motivated by the limitations of using either convolutional neural networks (CNNs) or transformers alone in the task of semantic segmentation of remote sensing images, a novel cross-model knowledge distillation framework, named distilling segmenters from CNNs and transformers (DSCT), is proposed in this paper to harness the complementary advantages of both models. The framework utilizes a channel-weighted attention-guided feature distillation (CAFD) module to condense the feature from the teacher model and enhance the student model’s focus on the teacher-focused regions. Additionally, a target-nontarget knowledge distillation (TNKD) module is proposed that decouples logit distillation into target and nontarget knowledge distillation to guide the student model in learning the underlying representations and decision boundaries from the teacher model. By learning the complementary knowledge from the teacher, our proposed DSCT framework improves the student’s segmentation performance without adding trainable parameters. Experiments on four available remote sensing datasets (ISPRS Potsdam, Vaihingen, GID and LoveDA) indicate that the proposed DSCT outperforms the state-of-the-art knowledge distillation methods and demonstrates its effectiveness and robustness.

Satellite video intrinsic decomposition has emerged as a promising area of research with significant application potential. However, existing methods still have certain limitations that hinder their effectiveness in extracting high-quality intrinsic information from complex scenes, ensuring temporal stability of the reflectance component, and achieving computational efficiency. In this article, an unsupervised satellite video intrinsic decomposition network (USVIDNet) is proposed, which overcomes the limitations encountered by existing methods. The USVIDNet incorporates three loss functions based on physical priors: reconstruction loss, chromaticity consistency loss, and spatiotemporal reflectance similarity loss, which provide constraints to guide the intrinsic decomposition process, eliminating the dependence on ground truth intrinsic images required for supervised learning. The network is based on a U-Net architecture variant, consisting of an encoder and two decoders. The encoder captures essential features of the input satellite video, while the decoders focus on predicting two components: reflectance and shading. To enhance the processing efficiency of intrinsic decomposition, a novel initialization-decomposition mode is proposed by leveraging the invariant background characteristics of staring satellites. Experiments are conducted on six Jilin-1 satellite videos to assess the performance of the proposed method in terms of intrinsic component extraction and improved ability of satellite video applications. The experimental results demonstrate the superiority of the proposed method.

Existing satellite video processing methods are mainly based on original video, ignoring the use of invariant background characteristics of staring satellites, and easy to be disturbed by rapid light changes. In order to improve application capability of satellite video, this paper establishes the satellite video intrinsic decomposition (SVID) model, including satellite video signal composition model, decomposition constraint with time-spatial unity similarity constraint, static and dynamic components separation by improving TRPCA, and decomposition acceleration based on reflectance transfer. With SVID, intrinsic decomposition and dynamic and static component separation are realized. Five Jilin-1 satellite videos are used to verify the validity, superiority and the potential applications of the proposed algorithm. By comparing with state-of-the-art intrinsic image decomposition method and intrinsic video decomposition method, the experimental results prove the superiority of the SVID method in extracting reflectance component. In addition, the experimental results also prove SVID has excellent application ability in scene background analysis and moving target tracking.

Due to interference with remote imaging by some natural factors, the multitemporal analysis ability is limited by the spectral drift between images. In this article, a new approach to optimize the existing multitemporal analysis system is proposed: multitemporal intrinsic image decomposition (MIID). The MIID method is designed to extract common spectral reflectance from multitemporal images. With MIID, multitemporal classification, changing detection, and index extracting will become extremely easy and more accurate. Firstly, without considering land cover change, the general MIID framework is proposed by adding local temporal–spatial energy constraints in traditional intrinsic images decomposition. On this basis, an improved MIID method with change detection (CD) (CD-MIID) capability is proposed to make the model adapt to the land cover change situation. Finally, specific steps of how to use MIID methods in the multitemporal analysis are given. Multitemporal multispectral/hyperspectral remote sensing images from GF-1, GF-2, GF-5, Landsat TM, and two groups of captured datasets with reflectance truth map are used to evaluate the performance. The experimental results show the following two points: first, the MIID methods achieve better extraction results of spectral reflectance. Second, the proposed MIID methods have better performance both on multitemporal classification and CD.

Satellite videos dynamically monitor the Earth’s surface by using staring imaging, which has gained increased attention and enabled target tracking applications. While various tracking methods are processed on the original video, the rapid light changes due to staring imaging are not considered and have a negative effect on tracking. To reduce the effects of illumination and improve the performance of satellite video target tracking, an intrinsic satellite video decomposition model with motion target energy constraint, called MTE-ISVD, is proposed in this paper. The proposed algorithm introduces two main constraints: The first is a temporal constraint of reflectance, which can solve the flicker problem by preserving reflectance coherence in the time domain with the property that the background pixels in satellite videos are nearly consistent between adjacent frames. The second is a motion target energy constraint, which can concentrate the signal energy of the motion targets in the reflectance by representing them with the surrounding background in the shading. The decomposition problem is reformulated as a quadratic function minimization, which can be addressed using the standard conjugate gradient in closed form. For visual and quantitative comparisons, we perform experiments on five Jilin-1 satellite videos and analyze the results in terms of visual comparison, target tracking improvement, stability evaluation and processing time comparison. The experimental results demonstrate that our proposed method outperforms the other representative intrinsic decomposition methods in terms of processing speed, stability, and motion target representation.

Satellite video scene classification (SVSC) is a challenging work in remote sensing. The main procedure of SVSC is spatial–temporal feature extraction. Unfortunately, massive numbers of dim small moving targets and the low signal-to-noise ratio (SNR) of satellite video bring great challenges to feature extraction. It is difficult to apply traditional feature extraction methods to SVSC because they are used to classify the actions of high-quality video. According to the theory of low-rank sparse decomposition, a Low-rank Sparse Representation Two-stream Network (LSRTN) is designed to increase the classification accuracy of two-stream networks. First, we propose a Low-rank Sparse Component Analysis Network (LSCAN) to decompose satellite videos into low-rank background images and sparse moving target sequences. The LSCAN possesses the advantage of low-rank sparse decomposition to solve small targets and has the capability to adjust the features using the data. Moreover, the LSCAN can efficiently improve the feature extraction of low SNR video. Second, a two-stream structure that was proven to be effective for multiclass video classification was applied to obtain the spatial features and temporal features in each stream. Finally, a fully connected layer integrates the features to classify the satellite video scenes. To utilize the label information, we refine the loss function to adjust the degree of low-rank sparse characteristics and ensure the classification accuracy of training. The experimental results demonstrate that the proposed method achieves better performance than the baseline methods for the SVSC task.

Aiming at accurately detect vehicles in high-resolution remote sensing images, this paper proposes a target detection framework combining region-based fully convolutional networks (R-FCN) and deformable convolution (DCN). The difficulty of vehicle detection is that its pixel range is small and difficult to detect, R-FCN calculates confidence scores pixel by pixel, and uses a confidence scoring map related to the number of categories and local parts of the target as the output of the network, which can make full use of the limited feature information of vehicles. As to the precision reduction caused by geometric deformation of vehicle images, the fixed structure of the convolution kernel is improved, and the convolution kernel of part of the convolution layers and region of interest (RoI) pooling layers in the network are deformable to make it adapt to the deformation of targets. Experiments show that the R-FCN equipped with deformable convolution and deformable RoI pooling has advantages in detection precision and detection time.

Weak target detection plays an important role in military and civilian fields. However, due to the limitation of the target size and the influence of complex background, the detection of weak target is a huge challenge. Therefore, based on high-resolution remote sensing image, this paper proposes a weak target detection network which combines super-resolution and deformable convolution. Firstly, the high-resolution remote sensing image is expanded and enhanced to eliminate the influence of complex background. Secondly, a detection network based on the deformable convolution and feature pyramid network (FPN) is used to solve the problem of less information caused by the fewer target pixels. In addition, this paper establishes a detection dataset only containing weak vehicles. The experimental results show that the proposed method achieves better detection results in the weak target detection problem.

Satellite videos provide rich dynamic information of observed scenes at a large spatial and temporal scale and will play an important role in the future space information network. This work devotes to revealing events of interest (EOI) from satellite video scenes by using a two-stream method. In satellite videos, individual frames reflect the static information like the basic scenes where the event was happening, while a sequence of frames determines the motion information. Considering these facts, a novel two-stream EOI detection framework is proposed, where one stream extracts static spatial information of satellite videos by AlexNet, whereas the other stream extracts the motion information using a local trajectories analysis method. First, the whole video scene is segmented into small spatial-temporal patches, where labeling EOI and non-EOI is completed. Next, the trajectories are extracted from 3-D satellite video cubes that are generated from event scene patches. Finally, this trajectory classification process is treated as a weak supervision learning problem and solved by sparse dictionary learning. The experimental results demonstrate that the proposed two-stream method is effective for EOI detection and has a huge potential for satellite video scenes analysis and understanding. The proposed method also outperforms the existing competitive models for video analysis.

Multitemporal domain adaptation (DA) is very useful for solving the spectral drift problem between different images and is a basis step of multitemporal classification. However, for high-resolution images, they always have a few spectral bands. A few spectral bands are difficult to establish accurate alignment model. In order to achieving accurate multitemporal alignment on a few spectral bands' high-resolution images, source label learning step is proposed in this letter and used to optimize traditional manifold alignment (MA). The core of this method is to improve the erroneous manifold structure by combining majority voting and weighting coefficients. Besides, this method is a universal step and can be used for optimizing all MA methods. Two groups of data sets captured by Chinese GF1 and GF2 satellites are used for performance evaluation. The experimental results demonstrate the effectiveness of our method and indicate our method significantly outperforms the traditional DA methods.

High-resolution (HR) remote sensing (RS) imaging opens the door to very accurate geometrical analysis for objects. However, it is difficult to simultaneous use massive HR RS images in practical applications, because these HR images are often collected in different multimodal conditions (multisource, multiarea, multitemporal, multiresolution, and multiangular) and learning method trained for one situation is difficult to use for others. The key problem is how to simultaneously tackle three main problems: spectral drift, spatial deformation, and band inconsistency. To deal with these problems, we propose an unsupervised tensorized principal component alignment framework in this paper. In this framework, local spatial-spectral patch data are used as basic units in order to achieve simultaneously multidimensional alignment. This framework seeks a domain-invariant tensor feature space by learning multilinear mapping functions which align the source tensor subspace with the target tensor subspace on different dimensions. In addition, an approach based on the Mahalanobis distance for dimensionality estimation of tensor subspace is proposed to determine best sizes of the aligned tensor subspace for reducing computational complexity. HR images from GF-1, GF-2, DEIMOS-2, WorldView-2, and WorldView-3 satellites are used to evaluate the performance. The experimental results show the following two points: first, the proposed alignment framework for multimodal HR images not only can align the different multimodal data more accurately than existing state-of-the-art domain adaptation methods, but also has a fast and simple procedure for large-scale data situation which is caused by HR imaging. Second, the proposed tensor dimensionality estimation method is an efficient technology for seeking the intrinsic dimensions of high-order data.

Multi-temporal remote sensing images classification have attracted more and more attention in the last decade because of a wide range of applications of multi-temporal images in long-term environmental monitoring and land cover change detection and increasing multi-temporal data available. At present, most papers investigated two temporal remote-sensing images classification. In fact, there is lots of distinctive information to be unexploited between two or more temporal images which can enhance classification effect and improve ability of detecting change area. In this paper, we present an evidential fusion framework of manifold alignment to combine more than two multi-temporal remote sensing images. Embedding of multi-groups two temporal images pairs after MA can be intergraded based a layered structure of D-S theory. The proposed method was evaluated using five Landsat 8 images. Results confirmed that the proposed algorithm performed better than those with only two temporal images.

Multitemporal Landsat images play an important role in remote sensing applications. Unfortunately, missing data caused by cloud cover and sensor-specific problems have seriously limited its application. To improve the usability of Landsat data, several recovery methods have been proposed to fill the missing values. But, current studies mostly focus on spatial dimension and ignore the continuity of data in time dimension. More importantly, multitemporal images have more potential than single image in selecting similar pixels for recovering the missing pixels. In this paper, to recover missing pixels by jointly utilizing multispectral and multitemporal information, tempo-spectral angle mapping (TSAM) is proposed at first to measure tempo-spectral similarity between pixels described in spectral dimension and temporal dimension. Then, a multitemporal replacement method is used to recover missing data with the pixel selected by TSAM. Two new indices are also proposed to evaluate the effectiveness of TSAM. Simulated and actual multitemporal scan-line corrector-off and cloud cover-Enhanced Thematic Mapper Plus images were used to assess the performance of our filling method. The quantitative evaluations suggest that the proposed method can predict the missing values accurately. The recovered results show that our method can keep the continuity of the boundary and is robust for the data with high percentage of missing.

In this paper, a new framework for tensor hyperspectral target detection is proposed. In this new framework, tensor is well integrated into the conventional target detection algorithm. As a result, a tensor matched subspace detector (MSD) for hyperspectral target detection is proposed. The proposed method is mainly applied to detect multipixel targets rather than subpixel targets. In this new method, the hyperspectral data are considered as a form of third-order tensor in order to jointly utilize the information of multidimensional data. In conventional detection methods, the spatial-spectral information has not been taken into account, even some algorithms have been presented for improving the utilization efficiency of the spatial-spectral structural feature, but the overall structural characteristic of the extracted feature is still ignored. In our algorithm, the tensor subspace projection is defined for the first time, which is easily calculated by three predetermined orthogonal direction mapping matrices without any iteration. Then, the test tensor blocks are projected into the tensor subspace and finally measured by the ratio of residual energy, just like the general likelihood ratio test. The proposed method can be regarded as an extension of conventional MSD. The reliability and superiority are demonstrated by the experiments on real hyperspectral imaging data sets. The experimental results indicate that our approach compares favorably to some classical and novel methods by jointly processing multidimensional data with tensorial form.

Since the scan line corrector (SLC) of the Landsat ETM+ sensor failed permanently in 2003, about 22% of the pixels in an SLC-off image are missed. Traditional gap filling methods always need a SLC-on image for reference, but the most similar sensor (Landsat TM) closed at 2011. And the potential of multi-temporal was also neglected in traditional filling methods. In this paper, a multi-temporal Landsat ETM+ gap filling method is proposed without using SLC-on reference which has ability to increase the utilization efficiency of multi-temporal images. The proposed method is mainly based on neighborhood similar pixel interpolator (NSPI) and the major contribution are find an effective way to select valid temporal and conjunctive use the temporal advantage to calculate of the target pixel value. Similarity both in spatial and temporal can be obtained in our method. Real multi-temporal Landsat data and missing gap location are used to assess the efficacy of the proposed method. Both qualitative and quantitative evaluations results suggest that our proposed method can predict the missing values very accurately and improve the utilization efficiency of multi-temporal.

Multiple kernel learning (MKL) is becoming more and more popular in machine learning. Traditional MKL methods usually learn the optimal combinations of both kernels and classifiers as the optimization task which is difficult to be solved. In this paper, we study a Boosting framework of MKL for classification in hyperspectral images. The multiple kernel Boosting (MKBoost) is proposed to solve the MKL problem, which apply the idea of Boosting to the multiple kernel classifiers based on the SVM. Experiments are conducted on different real hyperspectral data sets, and the corresponding experimental results show that MKBoost algorithm provides the best performances compared with the state-of-the-art kernel methods.