The weakly supervised semantic segmentation (WSSS) method aims to assign semantic labels to each image pixel from weak (image-level) instead of strong (pixel-level) labels, which can greatly reduce human labor costs. However, there are some problems in WSSS of remote sensing images such as how to locate labels accurately, and how to get precise segmentation edges. To address these issues, we propose a novel framework directly transferring the scene classification model to perform semantic segmentation. We first train a multi-label scene classification network as the encoder to obtain the pre-trained model, then the feature learned by the model is transferred to the decoder. Different from other methods, we propose a saliency map generator instead of the Class Activation Map for more accurate location information by making pixels belonging to the same class lie close together while different classes are separated in feature space. Meanwhile, we take the superpixel patch as processing unit to provide precise boundary inhibition for the saliency map. To assign semantic labels for each patch, combined with extracted salient region, we propose a module responsible for exploiting the consistency of spatial and semantic similarity between different patches. Finally, we incorporate the above two modules to supervise the training process of the decoder without generating pseudo labels as most methods do, thus simplifying the training process. Experimental results show that our method outperforms other weakly supervised approaches on DLRSD and WHDLD datasets with at least a 3% improvement on mean intersection over union.

Key regions are the similar regions of the scenes of the same category and they are the crucial and explicable areas. In this paper, we extract the key regions from the satellite images. The proposed method transfers the model of scene classification and obtains the result without the segmentation labels. First, the features of each layer of current scene classification models are extracted to find the relationships of key regions and scene classification labels. Second, we reconstruct the images from the relative features to obtain end-to-end results. There are areas of similar features between the images of the same categories and no areas of different categories. Last, we design the masks to obtain the key regions from the end-to-end results. The experiments were conducted on a typical scene classification dataset. The experimental result demonstrates the feasibility of extracting key regions by scene classification without relabeling the dataset.

Satellite video scene classification (SVSC) is a challenging work in remote sensing. The main procedure of SVSC is spatial–temporal feature extraction. Unfortunately, massive numbers of dim small moving targets and the low signal-to-noise ratio (SNR) of satellite video bring great challenges to feature extraction. It is difficult to apply traditional feature extraction methods to SVSC because they are used to classify the actions of high-quality video. According to the theory of low-rank sparse decomposition, a Low-rank Sparse Representation Two-stream Network (LSRTN) is designed to increase the classification accuracy of two-stream networks. First, we propose a Low-rank Sparse Component Analysis Network (LSCAN) to decompose satellite videos into low-rank background images and sparse moving target sequences. The LSCAN possesses the advantage of low-rank sparse decomposition to solve small targets and has the capability to adjust the features using the data. Moreover, the LSCAN can efficiently improve the feature extraction of low SNR video. Second, a two-stream structure that was proven to be effective for multiclass video classification was applied to obtain the spatial features and temporal features in each stream. Finally, a fully connected layer integrates the features to classify the satellite video scenes. To utilize the label information, we refine the loss function to adjust the degree of low-rank sparse characteristics and ensure the classification accuracy of training. The experimental results demonstrate that the proposed method achieves better performance than the baseline methods for the SVSC task.

Satellite videos provide rich dynamic information of observed scenes at a large spatial and temporal scale and will play an important role in the future space information network. This work devotes to revealing events of interest (EOI) from satellite video scenes by using a two-stream method. In satellite videos, individual frames reflect the static information like the basic scenes where the event was happening, while a sequence of frames determines the motion information. Considering these facts, a novel two-stream EOI detection framework is proposed, where one stream extracts static spatial information of satellite videos by AlexNet, whereas the other stream extracts the motion information using a local trajectories analysis method. First, the whole video scene is segmented into small spatial-temporal patches, where labeling EOI and non-EOI is completed. Next, the trajectories are extracted from 3-D satellite video cubes that are generated from event scene patches. Finally, this trajectory classification process is treated as a weak supervision learning problem and solved by sparse dictionary learning. The experimental results demonstrate that the proposed two-stream method is effective for EOI detection and has a huge potential for satellite video scenes analysis and understanding. The proposed method also outperforms the existing competitive models for video analysis.

Recently, super-resolution (SR) of satellite videos has received increasing attention as it can overcome the limitation of spatial resolution in applications of satellite videos to dynamic analysis. The low quality of satellite videos presents big challenges to the development of the spatial SR techniques, e.g., accurate motion estimation and motion compensation for multiframe SR. Therefore, reasonable image priors in maximum a posteriori (MAP) framework, where motion information among adjacent frames is involved, are needed to regularize the solution space and generate the corresponding high-resolution frames. In this article, an effective satellite video SR framework based on locally spatiotemporal neighbors and nonlocal similarity modeling is proposed. Firstly, local prior knowledge is represented by means of adaptively exploiting spatiotemporal neighbors. In this way, implicitly local motion information can be captured without explicit motion estimation. Secondly, the nonlocal spatial similarity is integrated into the proposed SR framework to enhance texture details. Finally, the locally spatiotemporal regularization and nonlocal similarity modeling bring out a complex optimization problem, which is solved via the iterated reweighted least squares in the proposed SR framework. The videos from the Jilin-1 satellite and the OVS-1A satellite are used for evaluating the proposed method. Experimental results show that the proposed method demonstrates better SR performance in preserving edges and texture details compared with the-state-of-art video SR methods.

Ship detection with very high resolution (VHR) remote sensing image has recently been an attractive topic due to rapid development of deep learning. Current researches on ship detection are generally confronted with a big challenge that existing methods failed to get high quality of object proposal with good intersection-over-union (IOU) before detection. In this paper, a Convolutional Neural Network (CNN) based renormalization method is proposed to improve the quality of object proposal. First, CNN is used to predict shape information of candidate ships' which are involved with rotation, location and scale in patches. Then, a renormalization net is designed to adjust the candidate ships in patches by correcting the shape information and renormalizing it to uniform patch. In this way, good candidate objects in patches could be generated and will be helpful with improving following ship detection. The proposed renormalization net was tested on a Google-Earth handcraft dataset. The experimental result demonstrates the proposed renormalization net greatly improve the ship detection with both of good detection accuracy and high IOU.