Traditional CNN-based training for instance segmentation is time-consuming owing to large datasets and complex network modules, making direct searching of architecture challenging. In this paper, we introduce an efficient framework, named EASInst. It can discover practical backbone and encoder architectures for the improved sparse activation instance segmentation model. Specifically, we construct a supernet for both backbone and encoder modules of SparseInst based on a differentiable method. In addition, kernel sharing mask and channel pruning technology are employed. Moreover, Taylor-Loss and a novel DY-Loss are devised for instance segmentation to improve the accuracy. Experiments show that the searched architectures outperform the existing Resnet-based real-time instance segmentation methods, which achieve 38.5 mAP with 39.5 FPS on COCO test-dev set.

Deep learning has found many applications in high-resolution remote sensing image interpretation. In this study, an image analysis system is presented, consisting of image segmentation and mineral volume change estimation. In this system, a revised U-Net structure, called Res-U-Net, is proposed by combining U-Net and residual structure for image segmentation. Experiments are performed on the collected high-resolution remote sensing images, which were annotated with iron ore positive, iron ore negative, and background, and the results demonstrate the superiority of the proposed Res-U-Net over other image segmentation methods. Our proposed Res-U-Net outperforms the traditional U-Net by achieving pixel-wise accuracy of 92% and mean intersection over union (mIOU) 86%, as well as faster frame rate of 35 FPS on test dataset.

Full-polarimetric synthetic aperture radar (SAR) images have the ability to provide physical patterns of the earth observation, no more than geometric information. In order to learn physical patterns from non-full-polarimetric SAR images, a complex-valued CNN is leveraged to learn a model containing physical parameters. The parameters are learned from the original complex scattering matrix of full-polarimetric SAR images and they can be adopted to extract physical patterns from non-full-polarimetric SAR images. Cloude and Pottier's H-α division, as the annotation principle, is computed by way of coherence matrix. We perform experiments on (German Aerospace Center) DLR's full-polarimetric, airborne F-SAR data, demonstrating that extracting physical patterns from non-full-polarimetric images is feasible. The comparative results illustrate that: 1) The best physical categoric patterns can be extracted from HV and VH polarimetric images in general, while performance from HH and VV polarimetric images are limited; 2) Cross-polarimetric SAR images have greater ability for surface and volume scattering, while co-polarimetric ones are better for multiple scattering extraction.

Recently, deep learning-based methods have brought new ideas for ship detection in synthetic aperture radar (SAR) images. However, several challenges still exist: 1) deep models contain millions of parameters, whereas the available annotated samples are not sufficient in number for training. Therefore, most deep detectors have to fine-tune networks pre-trained on ImageNet, which incurs learning bias due to the huge domain mismatch between SAR images and ImageNet images. Furthermore, it has a little flexibility to redesign the network structure; and 2) ships in SAR images are relatively small in size and densely clustered, whereas most deep detectors have poor performance with small objects due to the rough feature map used for detection and the extreme foreground–background imbalance. To address these problems, this paper proposes an effective approach to learn deep ship detector from scratch. First, we design a condensed backbone network, which consists of several dense blocks. Hence, earlier layers can receive additional supervision from the objective function through the dense connections, which makes it easy to train. In addition, feature reuse strategy is adopted to make it highly parameter efficient. Therefore, the backbone network could be freely designed and effectively trained from scratch without using a large amount of annotated samples. Second, we improve the cross-entropy loss to address the foreground–background imbalance and predict multi-scale ship proposals from several intermediate layers to improve the recall rate. Then, position-sensitive score maps are adopted to encode position information into each ship proposal for discrimination. The comparison results on the Sentinel-1 data set show that: 1) learning ship detector from scratch achieved better performance than ImageNet pre-trained model-based detectors and 2) our method is more effective than existing algorithms for detecting the small and densely clustered ships.

Single- and dual-polarimetric synthetic aperture radar (SAR) images provide very limited capabilities to interpret physical radar signatures. For generality and simplicity, we call single-polarimetric, dual-polarimetric, and fully polarimetric SAR (PolSAR) images flexible PolSAR images. In order to sufficiently extract physical scattering signatures from this kind of data and explore the potentials of different polarization modes on this task, this paper proposes a contrastive-regulated convolutional neural network (CNN) in the complex domain, attempting to learn a physically interpretable deep learning model directly from the original backscattered data. To achieve a better deep model containing physically interpretable parameters, the objective cost is compared to and selected from several commonly used loss functions in the complex form. The required ground-truth labels are generated automatically according to Cloude and Pottier's H-alpha division plane, which significantly reduces intensive labor cost and transfers this method to an unsupervised learning mechanism. The boundaries between different scattering signatures, however, sometimes show an erroneous separation. With the aim of aggregating intra-class instances and alienating inter-class instances, meanwhile, a complex-valued contrastive regularization term is computed mathematically and is added to the objective cost by a tradeoff factor. Moreover, data augmentation is applied to relieve the side effects caused by data imbalance. Finally, we performed experiments on German Aerospace Center's (DLR)'s L-band, high-resolution (HR), and airborne F-SAR data. Our results demonstrate the possibility of extracting physical scattering signatures from flexible PolSAR images. Physically interpretable potentials of SAR images with different polarization modes are analyzed, and we conclude with physical signature identification.

Fast multiclass object detection for remote sensing images plays an important role for a wide range of applications. Traditional methods based on a sliding window search lead to heavy computational costs and are unsuitable for multiclass detection. Recently, deep learning algorithms, especially faster region based convolutional neural networks (Faster R-CNN), which adopt a region proposal paradigm to avoid exhaustive search, has achieved state-of-the-art multiclass detection performance in computer vision. This paper investigates the use of Faster R-CNN in the earth observation community. We have three contributions: 1) It's the first time to successfully use Faster R-CNN for object detection in remote sensing images. It achieved faster speed (22 ×faster) and better performance (a mAP of 78% vs. 72%) than traditional methods; 2) we adopt data augmentation to train Faster R-CNN with limited samples; 3) we successfully tested our method on large-scale google earth images, which shows robustness of our method.

Synthetic Aperture Radar (SAR) image land cover classification is an important task in SAR image interpretation. Supervised learning, such as Convolutional Neural Network (CNN), demands instances which are accurately labeled. However, a large amount of accurately labeled SAR images are difficult to produce. In this paper, a Probability Transition CNN (PTCNN) is proposed for patch-level SAR image land cover classification with noisy labels. Firstly, deep features are extracted by a CNN model, followed by a probabilistic transition model, where true labels are treated as hidden variables and the posterior probabilities of true labels are transferred into their noisy versions. The whole network is trained with Caffe in a uniform fashion and a land cover database is used to produce noisy labels, which are randomly chosen with various proportions. Experimental results demonstrate that the proposed PTCNN model is robust to noise and gives a promising classification performance. Therefore, the PTCNN model may lower the standards for the quality of image labels, and shows its availability in practical applications.

Fine-scale classification in form of object extraction or segmentation for high resolution SAR images is a challenging task due to the existing local noises, object deformation and part missing. A novel SAR classification method based on CRFs which combines low-level features, label context and object structure priors is presented in this paper. Local label pattern is proposed in this paper to model the object structures by measuring the local label configuration on the grid layer of SAR images. We build a new CRFs model with label context and object structure priors for image classification. Besides, we adopt Mean Field approximation for efficient inference of our CRFs model. This work intends to implement an efficient classification framework by integrating high-level label context and object priors and apply it to fine-scale object extraction of SAR images. The framework demonstrates good performance in both accuracy and efficiency for object extraction or segmentation of simulated images and high resolution SAR images.

In this paper, we propose a fast PolSAR image superpixel segmentation method. This method takes a simple coarse-to-fine optimization technique to minimize a Markov-Random-Field (MRF) like energy function which integrates the Pol- SAR image statistic, spatial position and boundary smoothing. It updates boundary of superpixels staring with a large block level and iterates down to the final pixel level. We demonstrate the performance of our approach both on the synthetic and real full polarimetric images , showing that our proposed approach can achieve significantly faster convergence than SLIC method, and make a good compromise between accuracy and computation speed.

Convolutional Neural Network (CNN) has attracted much attention for feature learning and image classification, mostly related to close range photography. As a benchmark work, we trained a relatively large CNN to classify SAR image patches into five different categories, where the image patches tiled and annotated from a typical TerraSAR-X spotlight scene of Wuhan, China. The neural network designed in this paper consists of seven layers, including one input layer, two convolutional layers where each followed by a max-pooling layer, as well as two fully-connected layers with a final five-class softmax. Using the toolkit caffe, we achieved the training and testing accuracy of 85.7% and 85.6% respectively, which is considerably better than the traditional feature extraction and classification based SVM method and shows great potential of CNN used for SAR image interpretation. In order to accelerate the training process, a very efficient GPU implementation was employed.