Light field (LF) cameras capture the light rays of a 3D scene from multiple views simultaneously, and thus provide a more immersive experience of the real world as compared to traditional cameras. Although significant progress has been made in various LF image processing tasks, it remains challenging to effectively model the non-local spatial-angular correlations inherent in LF images, particularly when dealing with complex disparity variations. In this paper, we focus on orthogonal epipolar geometry of LF images and propose a generic Epipolar Transformer mechanism that incorporates geometrically meaningful correlations along the epipolar lines. Our Epipolar Transformer mechanism enjoys the following benefits: learning effective and diverse LF feature representations, delivering satisfactory results without redundant architectural designs, and enabling flexible extension to various LF-related tasks with simple adaptations. For LF spatial and angular super-resolution, our methods not only achieve state-of-the-art performance on benchmark datasets, but also demonstrate superior and robust performance on large disparity variations. For disparity estimation, we explore the use of geometry information encoded in our Epipolar Transformer to directly regress the disparity results, effectively avoiding the limitation of a fixed maximum disparity.

Multitask learning with a pretext task has excelled in time-series classification task lacking labeled data. The key to multitask learning is to build a pretext task and learn the most representative feature from the raw time series. In this article, we propose trend and order features for semi-supervised time-series classification via multitask learning (TOFL). Specifically, we propose a simple but effective pretext task-self-sequence order prediction (SOP)-to discover the order relation. In addition, we design a gradual trend fusion (GTF) block concatenating different trend features as the shared backbone network basis element to obtain high-quality trend features for the SOP task. Finally, we not only theoretically analyze the uniform stability and generalization error of TOFL but also evaluate the results compared with state-of-the-art (SOTA) supervised and semi-supervised methods on the 128 UCR datasets and three real-world datasets. TOFL demonstrates a high level of competitiveness and, in most cases, closely matches or even surpasses SOTA methods in terms of accuracy. The source code and data of TOFL are freely available at: https://github.com/Sample-design-alt/TOFL.

Compute-in-Memory (CIM) and weight sparsity are two effective techniques to reduce data movement during Neural Network (NN) inference. However, they can hardly be employed in the same accelerator simultaneously because CIM requires structural compute patterns which are disrupted in sparse NNs. In this paper, we partially solve this issue by proposing a bit level weight reordering strategy which can realize compact mapping of sparse NN weight matrices onto Resistive Random Access Memory (RRAM) based NN Accelerators (RRAM-Acc). In specific, when weights are mapped to RRAM crossbars in a binary complement manner, we can observe that, which can also be mathematically proven, bit-level sparsity and similarity commonly exist in the crossbars. The bit reordering method treats bit sparsity as a special case of bit similarity, reserve only one column in a pair of columns that have identical bit values, and then map the compressed weight matrices into Operation Units (OU). The performance of our design is evaluated with typical NNs. Simulation results show a 61.24 % average performance improvement and$1.51 \times-2.52 \times$energy savings under different sparsity ratios, with only slight overhead compared to the state-of-the-art design.

Visible-thermal small object detection (RGBT SOD) is a significant yet challenging task with a wide range of applications, including video surveillance, traffic monitoring, search and rescue. However, existing studies mainly focus on either visible or thermal modality, while RGBT SOD is rarely explored. Although some RGBT datasets have been developed, the insufficient quantity, limited diversity, unitary application, misaligned images and large target size cannot provide an impartial benchmark to evaluate RGBT SOD algorithms. In this paper, we build the first large-scale benchmark with high diversity for RGBT SOD (namely RGBT-Tiny), including 115 paired sequences, 93 K frames and 1.2 M manual annotations. RGBT-Tiny contains abundant objects (7 categories) and high-diversity scenes (8 types that cover different illumination and density variations). Note that, over 81% of objects are smaller than 16×16, and we provide paired bounding box annotations with tracking ID to offer an extremely challenging benchmark with wide-range applications, such as RGBT image fusion, object detection and tracking. In addition, we propose a scale adaptive fitness (SAFit) measure that exhibits high robustness on both small and large objects. The proposed SAFit can provide reasonable performance evaluation and promote detection performance. Based on the proposed RGBT-Tiny dataset, extensive evaluations have been conducted with IoU and SAFit metrics, including 30 recent state-of-the-art algorithms that cover four different types (i.e., visible generic object detection, visible SOD, thermal SOD and RGBT object detection).

Hyperspectral anomaly detection (HAD) intends to detect potential anomalous targets hidden in the background of hyperspectral images (HSIs) and has garnered substantial attention in various remote sensing photography and surveying applications. Recent research advances in the HAD domain have highlighted the significance of deep convolutional networks (DCNs) and vision transformers (ViTs)-based formulas. However, DCNs are long-range dependency-limited networks, whereas ViTs bear the computational burden of quadratic complexity. Owing to their prominent nonlocal representations and linear complexity, Mamba-based approaches have drawn growing attention. Our study pioneers the integration of Mamba into HAD tasks, presenting CWIMamba, which introduces a novel cross-scale windowed integration state space model for considering the spatial distribution characteristics of the anomaly targets. Specifically, we devise a cross-scale windowed state space model (CSWSSM) to scan the spatial-spectral features based on the window-based bottleneck SSM with different scales. For better multiscale feature integration, a multiscale spatial-spectral feature adaptive integration (MS3FAI) method is explored to generate an intensified representation of multiscale feature interaction and fusion based on the elaborate adaptive spatial-spectral weighting scheme. Moreover, we also devised a Haar discrete wavelet transform convolution module (HDWTCM) to fully replenish the local informative representation and enhance the discriminative frequency characteristics between anomalies and background, introducing more inductive local features for accurate background reconstruction and anomaly suppression. Extensive experiments on five multifarious HAD datasets and seven indicators substantiate the state-of-the-art detection performance, demonstrating the effectiveness of CWIMamba.

Multiframe infrared small target (MIRST) detection in satellite videos has been a long-standing, fundamental yet challenging task for decades, and the challenges can be summarized as follows. First, the extremely small target size, highly complex clutter & noise and various satellite motions result in limited feature representation, high false alarms and difficult motion analyses. In addition, existing methods are primarily designed for static or slightly adjusted perspectives captured by short-distance platforms, which cannot generalize well to complex background motion in satellite videos. Second, the lack of a large-scale publicly available MIRST dataset in satellite videos greatly hinders the algorithm development. To address the aforementioned challenges, in this article, we first build a large-scale dataset for MIRST detection in satellite videos (namely IRSatVideo-LEO), and then develop a recurrent feature refinement (RFR) framework as the baseline method for satellite motion estimation and compensation. Specifically, IRSatVideo-LEO is a semi-simulated dataset with synthesized satellite motion, target appearance, trajectory, and intensity, which can provide a standard toolbox for satellite video generation and a reliable evaluation platform to facilitate algorithm development. For the baseline method, RFR is proposed to be equipped with existing powerful CNN-based methods for long-term temporal dependency exploitation and integrated motion compensation and MIRST detection. Specifically, a pyramid deformable alignment (PDA) module is proposed to achieve effective feature alignment, and a temporal-spatial–frequent modulation (TSFM) module is proposed to achieve efficient feature aggregation and enhancement. Extensive experiments have been conducted to demonstrate the effectiveness and superiority of our scheme. The comparative results show that ResUNet equipped with RFR outperforms the state-of-the-art MIRST detection methods. The dataset and code are available athttps://github.com/XinyiYing/RFR.

In Resistive Random Access Memory (RRAM) based compute-in-memory designs, the column current readout circuits still consume too much area and power overhead, even if plenty of methods have been proposed to optimize the circuits. To alleviate this problem, this paper presents a novel current readout circuit to sense multiply-and-accumulate (MAC) result of RRAM array. Specifically, the circuit first integrate the stabilized and proportionally mirrored MAC current on a small capacitor until it fires, then sample the integration time with a set of reference signals with different carefully designed delays, and finally code the sampled result into a digital value. The proposed readout circuit has fine stability due to simple and determined relationship among the inputs, the RRAM cells’ states, and the MAC current. Meanwhile, the proposed design can achieve accurate MAC result readout at low resistance switching ratios, for it employs a current compensation circuit to remove background current caused by high resistance state RRAM cells. Our design is implemented using 28nm CMOS technology with a read latency of 2.8ns and an area occupation of 267μm2/channel, which is 12.5% and 89% less than state of the art design. Its power consumption, 0.092mW/channel, is also less than most counterpart designs.

Hyperspectral anomaly detection (HAD) is one of the momentous technologies in the field of Earth observation and remote sensing monitoring. Profiting from puissant deep feature extraction abilities, deep convolutional networks (DCN) perform excellently in the HAD domain. Nevertheless, limited by the restriction of unique local receptive fields, DCN-based detection methods struggle to catch the long-range dependence from a global perspective. In contrast, vision transformers (ViTs) perform better in global feature extraction but still disregard the local dependence properties. To this end, we proposed a novel method entitled the global-to-local spatial-spectral awareness transformer (G2LSSAT) network, in which the global transformer block (GTB) and local transformer block (LTB) are deployed in sequence to capture deep reconstruction characteristics from the global view to the local view in a spatial-spectral domain. In particular, the GTB is designed to explore the global spatial-spectral characteristics that are dependent on a crossbar-based global sparse attention module. Furthermore, the global glanced image is divided into multiple local patches and the LTB is devised to learn the local spatial-spectral features supported by a patch-based local self-invisible attention module. In addition, considering that the abnormal pixels always be unexpectedly reconstructed with the conventional self-attention module in ViTs, we introduce a invisible diagonal mask (IDM), which is embedded into the LTB module, to overshadow each pixel itself in the receptive field and reconstruct itself based on global and local dependent spatial-spectral features. Extensive experimental results on six datasets illustrate the superiority of the proposed G2LSSAT compared with other state-of-the-art detectors.

Recently, impressive improvements have been achieved in general object detection. However, tiny object detection remains a very challenging problem since tiny objects only occupy a few pixels. Consequently, the label assignment strategies used in general object detectors are not suitable for tiny object detection, because these algorithms tend to assign few or even no positive samples for tiny objects. In this article, we propose a simple yet effective Gaussian assignment (GA) strategy to solve this problem. Specifically, we first model the bounding boxes as 2-D Gaussian distributions and then encode training samples with a threshold. This strategy can assign more high-quality positive samples for tiny objects and adjust the weight of positive samples to balance the contribution from different-size objects. Extensive experiments on four tiny object detection datasets show that the proposed strategy significantly and consistently improves the performance of single-stage tiny object detectors. In particular, with our strategy, we bridge the performance gap between single-stage and state-of-the-art multistage detectors on the AI-TOD dataset (24.2% versus 24.8% in mAP) while maintaining the inference speed. The code is available athttps://github.com/zf020114/GaussianAssignment.

Training a convolutional neural network (CNN) to detect infrared small targets in a fully supervised manner has gained remarkable research interests in recent years, but is highly labor expensive since a large number of per-pixel annotations are required. To handle this problem, in this paper, we make the first attempt to achieve infrared small target detection with point-level supervision. Interestingly, during the training phase supervised by point labels, we discover that CNNs first learn to segment a cluster of pixels near the targets, and then gradually converge to predict groundtruth point labels. Motivated by this “mapping degeneration” phenomenon, we propose a label evolution framework named label evolution with single point supervision (LESPS) to progressively expand the point label by leveraging the intermediate predictions of CNNs. In this way, the network predictions can finally approximate the updated pseudo labels, and a pixel-level target mask can be obtained to train CNNs in an end-to-end manner. We conduct extensive experiments with insightful visualizations to validate the effectiveness of our method. Experimental results show that CNNs equipped with LESPS can well recover the target masks from corresponding point labels, and can achieve over 70% and 95% of their fully supervised performance in terms of pixel-level intersection over union (IoU) and object-level probability of detection (Pd), respectively. Code is available at https://github.com/XinyiYing/LESPS.

Exploiting spatial-angular correlation is crucial to light field (LF) image super-resolution (SR), but is highly challenging due to its non-local property caused by the disparities among LF images. Although many deep neural networks (DNNs) have been developed for LF image SR and achieved continuously improved performance, existing methods cannot well leverage the long-range spatial-angular correlation and thus suffer a significant performance drop when handling scenes with large disparity variations. In this paper, we propose a simple yet effective method to learn the non-local spatial-angular correlation for LF image SR. In our method, we adopt the epipolar plane image (EPI) representation to project the 4D spatial-angular correlation onto multiple 2D EPI planes, and then develop a Transformer network with repetitive self-attention operations to learn the spatial-angular correlation by modeling the dependencies between each pair of EPI pixels. Our method can fully incorporate the information from all angular views while achieving a global receptive field along the epipolar line. We conduct extensive experiments with insightful visualizations to validate the effectiveness of our method. Comparative results on five public datasets show that our method not only achieves state-of-the-art SR performance but also performs robust to disparity variations. Code is publicly available at https://github.com/ZhengyuLiang24/EPIT.

Abstract Extracting robust feature representation is one of the key challenges for person re‐identification (ReID) task. Although convolution neural network (CNN)‐based methods have achieved great success, they still cannot handle the part occlusion and misalignment caused by limited receptive field. Recently, pure transformer models have shown its power in the person ReID task. However, current transformer models adopt patches of equal‐scale as input, and cannot solve the problem of cross‐scale interaction properly. To overcome this problem, an adaptive cross‐scale transformer from a perspective of the graph signal, named ACSFormer, is proposed. Specifically, the self‐attention module is first treated as an undirected fully connected graph. And then, “node variation” is introduced as an indicator to adaptively merge neighbourhood tokens. To the best of the authors’ knowledge, their ACSFormer is the first work to attempt to combine pure transformers and graph signal processing in the field of person ReID. Extensive evaluations are conducted on three person ReID datasets to validate the performance of ACSFormer. Experiments demonstrate that this ACSFormer performs on par with state‐of‐the‐art CNN‐based methods and consistently improves transformer‐based baseline, for example, surpassing ViT‐baseline by 2.5%, 2.7% and 4.8% mAP on Market1501, DukeMTMC‐reID and MSMT17, respectively.

Hyperspectral anomaly detection (HAD) is regarded as an indispensable, pivotal technology in remote sensing and earth science domains. Nevertheless, most existing detection approaches for anomaly targets flatten 3-D hyperspectral images (HSIs) with spatial and spectral information into 2-D spectral vector data, which virtually breaks up the internal spatial structure in HSIs and degenerates the detection performance. To this end, we directly consider the HSI data cube as a 3-D tensor and develop a novel tensor low-rank approximation (TLRA) detection algorithm to separate the sparse anomalous component from the background with low-rank characteristics. Then, in light of the multi-subspace structure in heterogeneous backgrounds, we utilize multiple subspace learning (MSL) theory to encode the background tensor with a coefficient tensor and corresponding dictionary tensor. In addition, considering that different singular values indicate different information quantities and should be penalized to different extents, we introduce a tighter tensor rank surrogate named the ϵ-shrinkage tensor nuclear norm (ϵ-TNN) to recover the low-rank component more accurately. Meanwhile, concerning the sparse anomaly target, thel2,1constraint is incorporated to represent the group sparsity of the abnormal component. Finally, an effective iterative optimization algorithm based on the alternating direction method of multipliers (ADMM) is devised to solve the proposed TLRA-MSL model. We conduct extensive experiments on six hyperspectral datasets to prove the effectiveness and robustness of our method. The experimental results illustrate that better detection performance is obtained using the proposed model compared with other state-of-the-art algorithms.

Abstract This study addresses the problem of the unsupervised pre‐training of video representation learning. The authors' focus is on two common approaches: knowledge distillation and self‐supervised learning. The insight provided is that knowledge distillation and the rapidly advancing self‐supervised technique can be mutually beneficial. Combining these two approaches, a unified framework of self‐supervised learning and image‐based distillation (SSID) for unsupervised video pre‐training is proposed. The effectiveness of SSID in comparison to both image‐based distillation methods and the existing self‐supervised pre‐training baseline is demonstrated. In particular, the authors' model leverages three signals from the unlabelled data. First, the authors distil from the classifier of a 2D pre‐trained model as a soft label. To regularize the training process, the authors then build a novel positive pair of contrastive learning on the representation of the 2D/3D model. Finally, a self‐supervised pretext task is introduced to enhance the authors' model to become aware of the temporal evolution. The authors' experiment results showed that the learnt features achieved the best performance when transferred to action recognition tasks on UCF101 and HMDB51, reaching increases of 2.4% and 1.9% compared to the existing unsupervised pre‐training model, respectively.

Infrared dim small target detection is regarded as a critical technology for the interpretation of space-based remote sensing images. In recent years, driven by deep learning technology and the surge of data, remarkable effects have been achieved for dim small target detection in infrared images. Nevertheless, the intrinsic feature scarcity and low signal-to-clutter ratio (SCR) characteristics pose tremendous challenges to deep learning-based detection methods. In this letter, we present a novel sub-pixel sampling cuneate network (SPSCNet) to detect dim small targets in infrared images. The overall model architecture is based on an end-to-end cuneate network with multiple groups of parallel high-to-low resolution subnetworks. Specifically, we design a multi-scale feature reweighted fusion (MSFRF) module to effectively fuse multi-scale feature maps which contain both low-level detail features and high-level semantics information. In addition, considering that the pooling operation may lose dim small targets with low SCR, we also exploit a sub-pixel sampling scheme to greatly retain the features of small targets. Moreover, to better test and verify the performance of the proposed method, we also develop an infrared dim small target (IDST) dataset to conduct more comparative experiments. Extensive experiments on the SIRST and IDST datasets illustrate that the proposed SPSCNet yields state-of-the-art performance in comparison with other detection algorithms.

Rotated object detection in aerial images has received increasing attention for a wide range of applications. However, it is also a challenging task due to the huge variations of scale, rotation, aspect ratio, and densely arranged targets. Most existing methods heavily rely on a large number of predefined anchors with different scales, angles, and aspect ratios, and are optimized with a distance loss. Therefore, these methods are sensitive to anchor hyperparameters and easily suffer from performance degradation caused by boundary discontinuity. To handle this problem, in this letter, we propose a dense anchor-free rotated object detector (DARDet) for rotated object detection in aerial images. Our DARDet directly predicts five parameters of rotated boxes at each foreground pixel of feature maps. We design a new alignment convolution module (ACM) to extract aligned features and introduce a pixels-intersection over union (PIoU) loss for precise and stable regression. Our method achieves state-of-the-art performance on three commonly used aerial objects datasets (i.e., DOTA, HRSC2016, and UCAS-AOD) while keeping high efficiency. Code is available athttps://github.com/zf020114/DARDet.

Inspired by the great success of deep neural networks in image classification, recent works use Recurrence Plots (RP) to encode time series as images for classification. RP provide rich texture information and construct long-term time correlations, which are effective supplements to the networks. However, RP cannot handle the scale and length variability of sequences. Moreover, RP have serious tendency confusion problem. They cannot represent the upward and downward trends of sequences effectively. In addition to the defects of RP, existing time series classification (TSC) networks cannot adapt to the various scales of discriminative regions of time series effectively. To tackle these problems, this paper proposes a method, named MSRP-IFCN. It is composed of two submodules, the Multi-scale Signed RP (MSRP) and the Inception Fully Convolutional Network (IFCN). MSRP are proposed to handle the defects of RP. They comprise three components, namely the multi-scale RP, the asymmetric RP and the signed RP. We first use the multi-scale RP to enrich the scales of images. Then, the asymmetric RP are constructed to represent long sequences. Finally, the signed RP images are obtained by multiplying the designed sign masks to remove the tendency confusion. Besides, IFCN is proposed to enhance the existing TSC networks in multi-scale feature extraction. By introducing the modified Inception modules, IFCN obtains extensive receptive fields and better extracts multi-scale features from the MSRP images. Experimental results on 85 UCR datasets indicate the superior performance of MSRP-IFCN. The visualization results further demonstrate the effectiveness of our method.

Light field (LF) image super-resolution (SR) aims at reconstructing high-resolution LF images from their low-resolution counterparts. Although CNN-based methods have achieved remarkable performance in LF image SR, these methods cannot fully model the non-local properties of the 4D LF data. In this paper, we propose a simple but effective Transformer-based method for LF image SR. In our method, an angular Transformer is designed to incorporate complementary information among different views, and a spatial Transformer is developed to capture both local and long-range dependencies within each sub-aperture image. With the proposed angular and spatial Transformers, the beneficial information in an LF can be fully exploited and the SR performance is boosted. We validate the effectiveness of our angular and spatial Transformers through extensive ablation studies, and compare our method to recent state-of-the-art methods on five public LF datasets. Our method achieves superior SR performance with a small model size and low computational cost. Code is available at.1

Ship detection in remote sensing images plays a crucial role in various applications and has drawn increasing attention in recent years. However, existing arbitrary-oriented ship detection methods are generally developed on a set of predefined rotated anchor boxes. These predefined boxes not only lead to inaccurate angle predictions but also introduce extra hyperparameters and high computational cost. Moreover, the prior knowledge of ship size has not been fully exploited by existing methods, which hinders the improvement of their detection accuracy. Aiming at solving the above issues, in this article, we propose a center-head point extraction-based detector (CHPDet) to achieve arbitrary-oriented ship detection in remote sensing images. Our CHPDet formulates arbitrary-oriented ships as rotated boxes with head points that are used to determine the direction. Also, a rotated Gaussian kernel is used to map the annotations into target heatmaps. Keypoint estimation is performed to find the center of ships. Then, the size and head point of the ships are regressed. The orientation-invariant model (OIM) is also used to produce orientation-invariant feature maps. Finally, we use the target size as prior to fine-tune the results. Moreover, we introduce a new dataset for multiclass arbitrary-oriented ship detection in remote sensing images at a fixed ground sample distance (GSD) that is named FGSD2021. Experimental results on FGSD2021 and two other widely used datasets, i.e., HRSC2016 and UCAS-AOD, demonstrate that our CHPDet achieves the state-of-the-art performance and can well distinguish between bow and stern. Code and FGSD2021 dataset are available athttps://github.com/zf020114/CHPDet.

Complex generalized Gaussian distribution (CGGD) is quite significant in synthetic aperture radar (SAR) modeling since original focused SAR data are complex-valued. However, the estimation method of the vital parameter of the CGGD, i.e., the shape parameter, is seldom studied. This letter proposes a fast shape parameter estimation method of the CGGD in SAR images. The proposed method is developed based on a concept in the complex signal processing field, i.e., complex signal kurtosis (CSK). Specifically, this letter provides an introduction to the CSK at first. Then, the relationship between the shape parameter and the CSK is elaborated. Finally, the estimation chain based on the relationship is proposed. Experimental results demonstrate that the proposed method outperforms the state-the-of-art, i.e., the maximum-likelihood (ML) method proposed by Novey et al. It works in a near-real-time fashion with good estimation precision, being much faster than Novey's method and achieving better performance in distinguishing different kinds of non-Gaussianity of typical SAR targets.

Recently, deep learning-based methods have brought new ideas for ship detection in synthetic aperture radar (SAR) images. However, several challenges still exist: 1) deep models contain millions of parameters, whereas the available annotated samples are not sufficient in number for training. Therefore, most deep detectors have to fine-tune networks pre-trained on ImageNet, which incurs learning bias due to the huge domain mismatch between SAR images and ImageNet images. Furthermore, it has a little flexibility to redesign the network structure; and 2) ships in SAR images are relatively small in size and densely clustered, whereas most deep detectors have poor performance with small objects due to the rough feature map used for detection and the extreme foreground–background imbalance. To address these problems, this paper proposes an effective approach to learn deep ship detector from scratch. First, we design a condensed backbone network, which consists of several dense blocks. Hence, earlier layers can receive additional supervision from the objective function through the dense connections, which makes it easy to train. In addition, feature reuse strategy is adopted to make it highly parameter efficient. Therefore, the backbone network could be freely designed and effectively trained from scratch without using a large amount of annotated samples. Second, we improve the cross-entropy loss to address the foreground–background imbalance and predict multi-scale ship proposals from several intermediate layers to improve the recall rate. Then, position-sensitive score maps are adopted to encode position information into each ship proposal for discrimination. The comparison results on the Sentinel-1 data set show that: 1) learning ship detector from scratch achieved better performance than ImageNet pre-trained model-based detectors and 2) our method is more effective than existing algorithms for detecting the small and densely clustered ships.

Recent studies have shown that complex information in single-channel synthetic aperture radar (SAR) imagery has practically always been underrated. This improves the perception of their potential for ocean monitoring. Based on the in-depth interpretation of complex signal kurtosis (CSK), this paper proposes a new ship detection method based on CSK in single-channel SAR imagery. The proposed method consists of two main parts, i.e., region proposal and target identification. The basic idea is to first detect potential ship locations based on the region proposal. Then, the final ship target is acquired based on the target identification. Compared to conventional methods based on detected products, e.g., the constant false alarm rate (CFAR), the proposed method has three advantages. First, CSK can take advantage of both non-Gaussianity and noncircularity, which is the fundamental concept distinguishing complex signal analysis from the real case. Second, the proposed method can be intrinsically free of false alarms caused by radio frequency interference (RFI). Finally, the proposed method can avoid missing detection in dense target situations. This methodology has been demonstrated over significant data sets acquired from Sentinel-1, TerraSAR-X, and Gaofen-3. These results validate that CSK is a vital indicator of ship detection. Complex information is expected to play a more important role in single-channel SAR imagery.

Complex information in single-channel synthetic aperture radar (SAR) imagery is seldom used. This is a common practice based on the conventional resolution theory. However, with the advent of high-resolution SAR sensors, information in the complex data has been found to be of significance for ocean applications. In particular, we note that there is a special type of instrumental artifact in Sentinel-1 images. It is rarely researched and may be attributed to radio frequency interference (RFI). It has similar intensity with ships and can degrade ocean interpretation performance severely. This paper proposes an innovative method to discriminate ships from RFIs based on noncircularity and non-Gaussianity. Among them, noncircularity is calculated based on the measure called normalized noncircularity, and non-Gaussianity is estimated based on the complex generalized Gaussian distribution. The discrimination rationale is analyzed in detail. The experimental procedure is based on Sentinel-1 interferometric wide swath products. Only cross-polarization data are tested since RFIs are quite weak in co-polarization data. It is found that noncircularity and non-Gaussianity can characterize and identify the difference between ships and RFIs. Ships present larger noncircularity and sup-Gaussianity while RFIs are found to exhibit quite low noncircularity and mainly show sub-Gaussianity. The proposed method achieves quite good performance. These results show that noncircularity and non-Gaussianity are extremely helpful complements for single-channel SAR imagery interpretation.

Fast multiclass object detection for remote sensing images plays an important role for a wide range of applications. Traditional methods based on a sliding window search lead to heavy computational costs and are unsuitable for multiclass detection. Recently, deep learning algorithms, especially faster region based convolutional neural networks (Faster R-CNN), which adopt a region proposal paradigm to avoid exhaustive search, has achieved state-of-the-art multiclass detection performance in computer vision. This paper investigates the use of Faster R-CNN in the earth observation community. We have three contributions: 1) It's the first time to successfully use Faster R-CNN for object detection in remote sensing images. It achieved faster speed (22 ×faster) and better performance (a mAP of 78% vs. 72%) than traditional methods; 2) we adopt data augmentation to train Faster R-CNN with limited samples; 3) we successfully tested our method on large-scale google earth images, which shows robustness of our method.

This paper proposes a ship detection method based on weighted support vector machines (SVM) and m-χ decomposition in compact polarimetric (CP) synthetic aperture radar (SAR) imagery. Firstly, the proposed method constructs the weighted feature vectors by extracting CP parameters. Each feature will be weighted by the ReliefF method. Then, ship targets in CP SAR imagery are detected by the weighted SVM classifier. Finally, false alarms are removed by scattering mechanism strength differences corresponding to three components of m-χ decomposition. NASA/JPL AIRSAR airborne quad-polarimetric (QP) data are used to simulate the CP data in the circular transmitlinear receive (CTLR) mode. Experimental results show that the method performs well in detecting ship targets, and can reject azimuth ambiguities.

Traditionally, phase content and information contained in the complex data in single-channel synthetic aperture radar (SAR) imagery is often discarded based on the conventional resolution theory. With the rapid development of SAR technology, however, ship target is no longer a point target but an extended target in high resolution SAR imagery. Thus, the conventional resolution theory is not strictly applicable to high resolution SAR imagery. Noncircularity can describe the distribution consistency between the real and imaginary parts. In this paper, we proposed a method using noncircularity parameters for ship detection in high resolution SAR imagery. The potential by using noncircularity parameters for ship detection is studied in detail. Experimental results based on TerraSAR-X data show that noncircularity parameters can identify ship targets well and can discriminate azimuth ambiguities. We believe that noncircularity parameters can benefit ship detection in various research aspects.

This paper proposes a coupled region based convolutional neural networks (R-CNN) to automatically detect vehicles in aerial images. Traditional methods are mostly based on sliding-window search, and use handcrafted or shallow-learning based features. They have limited description ability and heavy computational costs. Recently, a series of R-CNN based methods have achieved great success in general object detection. Inspired by the previous work, we propose a coupled R-CNN to detect small size vehicles in large-scale aerial images. First, a vehicle proposal network (VPN) is proposed to generate candidate vehicle-like regions, using a hyper feature map combined by feature maps of different layers. Then, a vehicle classification network (VCN) is developed to further verify the candidate regions and classify vehicles in eight directions. In this study, our method is tested on a challenge Munich vehicle dataset and the collected vehicle dataset, with improvements in accuracy and speed compared to existing methods.

This paper proposes a PolSAR imagery unsupervised classification framework based on consensus similarity network fusion (CSNF), which is generally utilized for biomedical Sciences and for the first time used for PolSAR imagery classification in our work. First, the PolSAR image is divided into superpixels by a fast superpixel segmentation method and five groups of feature vectors are extracted based on the superpixels. Second, CSNF is performed on the five affinity matrixes constructed from the five groups of feature vectors to obtain a fused similarity matrix. Third, spectral clustering based on the fused similarity matrix is adopted to automatically achieve the classification results. Finally, a postprocessing procedure based on dissimilarity measure is performed to smooth the classification results and correct the misclassified pixels. The experimental results conducted on both a simulated PolSAR image and a real-world PolSAR image show the superiority of the proposed method.

In the detection applications of synthetic aperture radar (SAR) data, a crucial problem is developing precise models for clutter statistics. Generalized gamma distribution (GΓD) has been widely applied in many fields of signal processing, and it has been demonstrated to be an appropriate model for describing the statistical behaviors of SAR sea clutter, wherein parameter estimation is a key issue for determining the practical application of GΓD. Work that contains three major aspects is performed in this paper. First, an approximate estimator for GΓD parameters based on the well-known “method-of-log-cumulants” is derived; a theoretical comparison between the approximate estimator and other known estimators is also presented. Second, based on this estimator, a scheme of parameter estimation is further given by comprehensively considering estimation precision, speed, and applicable conditions. The simulation results show that the presented scheme is fast and effective. Third, we assess the fitting performance of GΓD and the proposed scheme using real SAR sea clutter data, and compare the model with generalized-K distribution. The experiments on single-look complex and multilook processing L-band ALOS-PALSAR and C-band RADARSAT-2 SAR data verify the effectiveness of the proposed scheme of GΓD parameter estimation. Moreover, several examples of ship detection in real SAR images testify to the usefulness of the proposed scheme in practical applications.

In this paper, an efficient multi-focus image fusion approach is proposed based on scale vector norm of image nonsubsampled contourlet transform(NSCT). According to the analysis of multi-focus imaging mechanism, the defocused optical imaging system can be characterized as a Gaussian low pass filter. Combining with the NSCT of an image, whether a region is in focus or out of focus can be determined by its corresponding high frequency coefficients' energy. Therefore, a scale vector norm is put forward in this paper and the fusion principles for different sub-band coefficients are also presented based on the regional scale vector norm. Experimental results demonstrate the proposed method can extract more details from multi-focus source images at a large extend and achieve more satisfactory results compared with other NSCT based methods.

In this paper, we address the problem of semi-supervised visual domain adaptation for transferring scene category models from ground view images to overhead view very high-resolution (VHR) remote sensing images. We introduce a multiple kernel learning domain adaptation algorithm to fuse the information from multiple features and cope with the considerable variation in feature distributions between images from two domains. For each image, we first extract eight state-of-art local features and use the pretrained scene attribute model from ground-level SUN attribute database to predict attribute labels. For each scene class we learn an adapted target classifier based on multiple feature kernels by minimizing both the structural risk functional and the mismatch between data distributions of two domains. Experimental results demonstrate that it is possible to use a scene category model learned on a set of ground view scenes for semi-supervised classification of VHR remote sensing images.

Land masking is one of the most important stages for ship detection in synthetic aperture radar (SAR) images. However, a fast and efficient algorithm for land masking in SAR images is far from resolved. Current land masking algorithms are time-consuming or not accurate enough for ship detection in SAR images. In this paper, an algorithm for land masking is proposed. It is designed for ship detection in SAR images based on a series of image processing steps. Experimental results based on real SAR data demonstrate that the algorithm proposed in this paper is fast and accurate enough for ship detection in SAR images.

With the rapid development of spaceborne Synthetic Aperture Radar (SAR) and the increasing need of ship detection, research on adaptive ship detection in spaceborne SAR imagery is of very great importance. Focusing on practical problems of adaptive ship detection, this paper present a highly adaptive ship detection method for spaceborne SAR imagery. It applies two different detection strategies to high and low resolution SAR imagery respectively. By taking into account the imaging mode, incidence angle, polarization channel of SAR imagery, it implements the adaptive ship detection in spaceborne SAR imagery. Experimental results based on real data show that the proposed method is able to detect all ship targets adaptively in a real-time fashion.

In image processing, the detection of keypoint plays a significant role in foundation work of many image applications. Most widely distributed features in images are corners. Among the famous corner detectors, Harris detector has shown its excellent performance. However, when coming up with SAR image, the speckle noise may severely influence the performance of Harris. And Harris is seldom used in SAR image processing. In this paper, we take advantage of the premium properties of the Bilateral Filter which is robust to speckle noise while preserving the details. Then we propose a new corner detector called bf-Harris. We study the performance of the proposed detector and compare it to several existing approaches. The result shows the algorithm has an excellent performance.

Speckle existed in SAR image is an undesirable product of specific imaging principle which influences SAR image interpretation and processing. In this paper, a new SAR image denoising algorithm has been proposed combining cluster with sparse representation under the non-local methodology. Due to the similar clustered patches, the sparsity coding of clustered patches is sparser. And clustered patches with similar structure could have the same constraint condition defined by the center of clustering. Thus, the non-local patches are clustered and filtered as a whole with shrinked sparsity coding. This algorithm has preferable denoising results on both simulated images and real SAR images. Experiments show prospects with speckle of different degrees compared with state-of-the-art despeckling methods. Proposed algorithm performs well both in noise reduction and detail preservation.

The simple linear iterative clustering (SLIC) method is a popular recently proposed superpixel algorithm. However, it may provide bad superpixels for the synthetic aperture radar (SAR) images due to the influence of speckle and large dynamic range of pixel intensity. In this paper, an improved SLIC algorithm for SAR images is proposed by employing the probability density function (PDF) information of SAR image pixel clusters. In this algorithm, a local clustering scheme combining data similarity with spatial proximity is designed, instead of the local k-means clustering used in the standard SLIC method. Moreover, for the post-processing, an edge evolving scheme with a local Bayesian criterion is introduced, instead of the connected components algorithm. In addition, for the precise statistical modeling of SAR images, the generalized gamma distribution (G?D) is exploited. Finally, the superiority of the proposed algorithm is validated on both simulated and real-world SAR images.

Bilateral filtering is a technique to smooth images while preserving edges; it employs both geometric closeness and intensity similarity of neighbouring pixels. When intensity similarity of neighbouring pixels is very high, however, bilateral filtering weakens into Gaussian filtering. The performance does not improve significantly while the computation is still expensive. Many existing accelerated algorithms, however, ignored this basic fact. In this study, a hybrid bilateral filtering algorithm based on edge detection is proposed. By making use of edge detection, the proposed algorithm combines bilateral filtering and Gaussian filtering and its degree can be controlled by a threshold. Experimental results show that the proposed algorithm is able to reduce the computation efficiently and achieve better performance. What is more, the proposed algorithm shows potential to speed up existing accelerated bilateral filtering algorithms.

We address the problem of unsupervised visual domain adaptation for transferring scene category models and scene attribute models from ground view images to overhead view very high-resolution (VHR) remote sensing images. We introduce a discriminative cross-view subspace alignment algorithm where each view is represented by a subspace spanned by eigenvectors. The source subspace is created using partial least squares correlation, whereas the target subspace is constructed by principal component analysis. Then, a mapping that aligns the source subspace and the target subspace is learned by minimizing a Bregman matrix divergence function. Finally, we project the labeled source data into the target aligned source subspace and the unlabeled target data into the target subspace and perform classification. Experimental results demonstrate that it is possible to use a scene category model or a scene attribute model learned on a set of ground view scenes for classification of VHR remote sensing images. Furthermore, the transferred visual attribute-based representations are human understandable and the classification results are better or comparable with state-of-the-art methods.

The product of multilook amplitudes (PMA) detector has been used to detect ships in high-resolution dual-polarization synthetic-aperture-radar images. However, the adaptive constant false-alarm rate (CFAR) technique of the PMA detector is desirable for practical applications, wherein a crucial problem is to find an appropriate model to describe the PMA statistics for varied sea surfaces. First, we consider a new probability density function to characterize the PMA statistics of homogeneous sea surfaces. Second, by using the new density and multiplicative model, the PMA detector's statistical model for nonhomogeneous sea surfaces is specified and demonstrated to be the G0distribution. Then, a theoretical analysis of the relationship between the performance of the standard CFAR detection and the parameters in the G0distribution is conducted. Experiments performed on the measured RADARSAT-2 and NASA/JPL AIRSAR images verify the effectiveness and appropriateness of the G0model for describing the statistical behavior of the PMA of sea clutter, as well as the usefulness of the model for practical ship-detection applications.

For the classification of synthetic aperture radar (SAR) images, traditional pixel-based Bayesian classifiers suffer from an intrinsic flaw that categories with serious overlapped probability density functions cannot be well classified. To solve this problem, in this letter, a region-based classifier for SAR images is proposed, where regions, instead of individual pixels, are treated as elements for classification. In the algorithm, each region is assigned to the class that minimizes a criterion referring to the Kullback-Leibler distance. Besides, the generalized gamma distribution (GΓD), a flexible empirical model, is employed for the statistical modeling of SAR images. Finally, with a synthetic image and an actual SAR image acquired by the EMISAR system, the effectiveness of the proposed algorithm is validated, compared with the pixel-based maximum-likelihood method and two region-based Bayesian classifiers.

This letter proposes a novel segmentation algorithm for synthetic aperture radar (SAR) images based on hierarchical region merging and edge evolving. To cope with the influence of speckle in SAR images, a statistical stepwise criterion, the loss of log-likelihood function (LLF) of image partition, is utilized for region merging. For this merging procedure, precise distributions of image partitions are essential, and we employ the generalized gamma distribution (GΓD) for modeling SAR images. Besides, the traditional region merging methods often suffer from the initial image partition that may lead to coarse segment shapes. It motivates us introducing a novel edge evolving scheme into the segmentation algorithm. It consists of two iterative steps: the evolution of edge pixels with a maximum likelihood (ML) criterion and that with a maximum a posterior (MAP) criterion using a Markov random field (MRF) model. The performance of the proposed algorithm is validated on two actual SAR images from the AIRSAR and EMISAR systems.

In this letter, a novel constant false alarm rate (CFAR) detection algorithm for high-resolution synthetic aperture radar (SAR) images is proposed with the generalized gamma distribution (GΓD) modeling the background. At first, the method of log-cumulants is introduced for estimating the parameters of the GΓD. In addition, a closed-form expression for the detection threshold of the CFAR algorithm is derived, which refers to the inverse incomplete gamma function. Finally, comparing with the algorithms using the Weibull,KA, andGA0distributions for background, the advantages of the proposed algorithm, including maintaining the false alarm rate and the efficiency, are validated with an actual high-resolution SAR image.

High resolution SAR imagery captures both the sea background and ship target more explicitly. This paper proposed an algorithm based on EXS-C-CFAR (excision-switching context based CFAR) and Alpha-stable distribution to detect ships in high resolution SAR imagery. From experiment results, it is derived that the Alpha-stable distribution models spiky sea clutter well and the EXS-C-CFAR has good ship detection performance on JPL/NASA AIRSAR data. Moreover, context information utilized in the detector preserves more ship structures.