Cross-scene hyperspectral image classification aims to enable the model to complete the classification of unlabeled target domain data by learning from labeled source domain data. Aiming at the problem that most current cross-scene hyperspectral image classification algorithms do not fully consider the cross-domain feature representation and category decision boundary optimization, a cross-domain Feature Extraction and Category Decision collaborative optimization (FECD) network is proposed. First, an adaptive feature discovery based on dynamic masks is designed. In this mechanism, the dynamically scaled masks are applied to the 3D representation of source and target domain data to generate an informative feature space and enhance the cross-scene discrimination potential of the model. Second, a dual-stream convolutional cross-domain feature extraction based on Mamba stream and ViT stream is constructed. Long sequence modeling and convolutional attention mechanisms are used to capture cross-domain spectral features between pixel, and self-attention mechanisms and multi-scale convolution are used to excavate cross-domain space patterns of pixel. Finally, a category decision based on the co-optimization of dual-stream classifiers is implemented. The spectral and spatial boundaries learned by the dual streams are fused to optimize the category decision. Therefore, the risk of false labeling is avoided while obtaining more accurate category boundaries. Compared with seven state-of-the-art algorithms on three widely used datasets, FECD obtains better categorization results on three categorization metrics: OA, AA, and Kappa.

Partial multi-label image classification (PMLIC) learns from typical weak supervision, where each image is labeled with a set of candidate labels, only some of which are correct. We find that noisy labels generate conflicting gradient signals that disrupt the learning of latent true labels, causing the model to prefer learning clean negative labels that provide consistent supervisory signals, thereby hindering disambiguation. Meanwhile, noisy labels cause the model to activate misattributed pixel regions, which interfere with feature pattern extraction, leading to inaccurate label correlation. In this paper, we propose a PMLIC framework that constructs a correlation-induced negative suppression disambiguation loss (CoNeS). First, we exploit the property that networks tend to learn clean labels first by extracting class activation maps to identify and screen misattributed pixel regions. Meanwhile, we aggregate noise-disturbed feature patterns into more expressive representations via k-means clustering and construct accurate label correlations to aid disambiguation. In addition, we design the negative suppression disambiguation loss to focus the model on disambiguation by introducing a weight distribution to suppress the contribution of negative labels. This weighting distribution can be adaptively inferred by a closed-form solution. Extensive experiments demonstrate that the CoNeS framework achieves significant advantages over current state-of-the-art methods. Specifically, it achieves average mAP improvements of 1.26%, 2.74%, 0.85%, and 0.33% on the VOC 2007, MS-COCO, VG-256, and CUB-200 datasets at different resolutions and noise rates. Code has been made available at https://github.com/zhongjingyu1/CoNeS.

The anchor-based clustering method is currently a predominant technique for handling large-scale data. However, in multiview data, existing anchor-based methods face a key challenge: balancing individual anchor graph distinctiveness with final consistency. To address this challenge, we propose a large-scale multiview clustering (MVC) method via joint learning of anchor representation and multigraph alignment (ARMGA). Specifically, ARMGA introduces a unified framework that facilitates the concurrent learning of single-view anchor representations and virtual graph-based multigraph alignment. The approach aims to preserve the adaptability of anchor learning across different views, while ensuring the ultimate consistency of the merged anchor graph. Furthermore, ARMGA employs Schatten- $\boldsymbol {p}$ norm on the tensor formed by the adaptive anchor representation, originating from multigraph alignment, to reinforce cross-view consistency. This technique effectively leverages complementary information preserved across views to bolster the overall structure and consensus information. Ultimately, to attenuate the noise impact on the anchor representation matrix, ARMGA capitalizes on the cosine angle information from the low-rank representation as coefficients within the relationship matrix and efficiently reduces computational complexity through deductions. On nine datasets, ARMGA has exhibited a notable improvement in clustering performance indicators by 2%-10% over other algorithms, while also maintaining lower time complexity.

In recent years, attention mechanisms have been widely used in many fields due to their excellent image focusing ability to produce more discriminative feature representations. However, in human pose estimation, methods based on attention mechanisms tend to have high computational overhead and are difficult to process video data in real time. In addition, existing algorithms do not make good use of the similarity between consecutive frames, and often repeat the computation many times on the same image. Therefore, this paper proposes a method based on a local attention mechanism and temporal prediction. The method first passes the input image through a body detector and then focuses attention on the head of the person, generating a large module of head information perception. This helps to find all the people in the image, avoiding missed detections, and this approach, which uses a local attention mechanism, has a small computational overhead. Then, to make the network layers closer to each other while keeping the parameters sparse, the features generated by the deep network will be reconstructed and compared with the input image. Finally, in order to reduce repetitive computation, the method determines the similarity between the preceding and following frames by means of multiple sampling points, which is used to determine whether the information from the previous frame is used to guide the localization of the nodes in the following frame. This allows the algorithm to remain real time when processing video. Experiments on the COCO and PoseData I datasets show that the algorithm outperforms all 10 comparison algorithms in terms of both accuracy and image continuity.

Evolutionary computation (EC)-based neural architecture search (NAS) has achieved remarkable performance in the automatic design of neural architectures. However, the high computational cost associated with evaluating searched architectures poses a challenge for these methods, and a fixed form of learning rate (LR) schedule means greater information loss on diverse searched architectures. This paper introduces an efficient EC-based NAS method to solve these problems via an innovative meta-learning framework. Specifically, a meta-learning-rate (Meta-LR) scheme is used through pretraining to obtain a suitable LR schedule, which guides the training process with lower information loss when evaluating each individual. An adaptive surrogate model is designed through an adaptive threshold to select the potential architectures in a few epochs and then evaluate the potential architectures with complete epochs. Additionally, a periodic mutation operator is proposed to increase the diversity of the population, which enhances the generalizability and robustness. Experiments on CIFAR-10, CIFAR-100, and ImageNet1K datasets demonstrate that the proposed method achieves high performance comparable to that of many state-of-the-art peer methods, with lower computational cost and greater robustness.

Due to the domain shift, it is challenging to achieve ideal experimental results for cross-domain few-shot learning (FSL) in hyperspectral image (HSI) classification. Most existing FSL algorithms are impacted by the limited samples, and they do not effectively leverage the representations from different layers of the network. Therefore, this article proposes an FSL based on embedded self-distillation and adaptive Wasserstein (ESAW-FSL) distance for HSI classification. First, the embedding self-distillation network is proposed in the feature extraction process of the source domain (SD) and the target domain (TD). The embedding self-distillation network utilizes self-distillation from different perspectives to get discriminative features. In the SD, the mask evaluation of embedded features is employed to guarantee the learning of guiding features. Second, a domain adaptation based on adaptive Wasserstein distance is designed to alleviate the domain shift problem between the domains. A lightweight feature correlation network learns the comprehensive cost matrix in the Wasserstein distance adaptively, and the obtained cost matrix helps achieve domain adaptation by an iterative algorithm. Finally, a focal loss based on double softening is adopted in the process of FSL. The probability is double softened to improve the ratio of correctly classifying hard samples. Experiments are conducted on three widely used hyperspectral datasets and compared with six state-of-the-art algorithms. The overall accuracy (OA) and average accuracy (AA) are achieved in multiple experiments, demonstrating the effectiveness of ESAW-FSL.

Markov Random Fields (MRF) effectively capture local contextual information by modeling the spatial dependencies between pixels, which helps highlight details and enhances segmentation smoothness. To fully exploit MRF for synthetic aperture radar (SAR) image segmentation, we propose a novel edge-enhanced cascaded MRF (ECMRF) approach. Specifically, we introduce multiple edge-constrained filters to emphasize SAR image boundaries and provide relatively clean features. Building on this, we present a cascaded MRF framework that sequentially integrates region-level and pixel-level segmentation with feature perturbation and fusion to generate the final segmentation output. The framework comprises four key components: (1) a region-level MRF, regulated by edge features, to achieve precise region segmentation; (2) a pixel-level MRF with selective label smoothing to refine edges and reduce noise clusters; (3) equal-channel feature perturbation to increase feature diversity; and (4) a random probability-based feature fusion scheme to merge the input features. Experimental results demonstrate that our ECMRF outperforms six state-of-the-art comparable methods, underscoring its competitive performance.

The models currently used for Polarimetric Synthetic Aperture Radar (PolSAR) image classification tasks have problems such as complex network structures, poor distinction of detailed features, and fixed loss weights during the training process. In response to these problems, this paper proposes a PolSAR image classification method based on knowledge distillation using adaptive learning and channel amplification features. Firstly, this paper builds a knowledge distillation framework for PolSAR. Using a teacher network trained in advance that can acquire global knowledge to guide the student. This framework reduces the computational complexity and improves the classification accuracy of the student. Then, an adaptive loss weight learning mechanism is designed, which sets the weight of the Kullback-Leibler divergence loss during training into a learnable mode. The weight can be automatically adjusted according to the actual training situation of the student. Finally, a scheme for channel amplification to enhance features is proposed. This scheme obtains channel weights based on the student’s feature map information. These weights are amplified, strengthening the network’s ability to obtain feature information. Compared with the five PolSAR image classification algorithms, the method proposed in this paper uses lower computational complexity to obtain higher classification accuracy on the Flevoland, San Francisco, and Xi’an datasets.

Recently, cross-domain few-shot learning (FSL) has achieved remarkable performance in hyperspectral image classification (HSIC). However, current prototype-based FSL methods overlook the redundancy between prototypes, which may lead to intra-domain redundancy dependence and interfere with the effective alignment of data distributions between the source domain (SD) and target domain (TD). To address these issues, we propose a prototype mask contrast and cross-stage feature refinement (PMC-CFR) method for few-shot HSIC. Specifically, a neighbor-resistant multidimensional feature fusion network (NRMFF) is designed to fuse features inclined toward different dimensional information after random neighboring-pixel removal. Next, the parameter-free adaptive mask generator (PAMG) identifies potential redundant components based on the statistical distribution of intra-domain prototypes to form negative pairs, while treating other components as key features to form positive pairs. Contrastive pairs are further processed through graph convolution (GC) for supervised contrastive learning (SCL), which mitigates the dependence on intra-domain redundancy and enhances the learning of key features. To extract more representative features, multi-level representations are formed depending on whether the shared multilayer perceptron (MLP) is applied to focus on key components in each domain by fitting positive pairs. Meanwhile, the directional calibration (DC) separates inter-class prototypes across levels by balancing direction and distance. The synergy between MLP and DC in hierarchical feature modeling and semantic separation enables cross-stage feature refinement. Experimental results on four public HSI datasets demonstrate that the PMC-CFR outperforms many advanced contrastive methods, and feature visualizations further support its effectiveness in cross-domain feature distribution alignment.

Unsupervised domain adaptation (UDA) techniques, extensively studied in hyperspectral image (HSI) classification, aim to use labeled source domain data and unlabeled target domain data to learn domain invariant features for cross-scene classification. Compared to natural images, numerous spectral bands of HSIs provide abundant semantic information, but they also increase the domain shift significantly. In most existing methods, both explicit alignment and implicit alignment simply align feature distribution, ignoring domain information in the spectrum. We noted that when the spectral channel between source and target domains is distinguished obviously, the transfer performance of these methods tends to deteriorate. Additionally, their performance fluctuates greatly owing to the varying domain shifts across various datasets. To address these problems, a novel shift-sensitive spatial-spectral disentangling learning (S4DL) approach is proposed. In S4DL, gradient-guided spatial-spectral decomposition (GSSD) is designed to separate domain-specific and domain-invariant representations by generating tailored masks under the guidance of the gradient from domain classification. A shift-sensitive adaptive monitor is defined to adjust the intensity of disentangling according to the magnitude of domain shift. Furthermore, a reversible neural network is constructed to retain domain information that lies not only in semantic but also the shallow-level detailed information. Extensive experimental results on several cross-scene HSI datasets consistently verified that S4DL is better than the state-of-the-art UDA methods. Our source code will be available athttps://github.com/xdu-jjgs/IEEE_TNNLS_S4DL.

Multilabel feature selection solves the dimension distress of high-dimensional multilabel data by selecting the optimal subset of features. Noisy and incomplete labels of raw multilabel data hinder the acquisition of label-guided information. In existing approaches, mapping the label space to a low-dimensional latent space by semantic decomposition to mitigate label noise is considered an effective strategy. However, the decomposed latent label space contains redundant label information, which misleads the capture of potential label relevance. To eliminate the effect of redundant information on the extraction of latent label correlations, a novel method named SLOFS via shared latent sublabel structure and simultaneous orthogonal basis clustering for multilabel feature selection is proposed. First, a latent orthogonal base structure shared (LOBSS) term is engineered to guide the construction of a redundancy-free latent sublabel space via the separated latent clustering center structure. The LOBSS term simultaneously retains latent sublabel information and latent clustering center structure. Moreover, the structure and relevance information of nonredundant latent sublabels are fully explored. The introduction of graph regularization ensures structural consistency in the data space and latent sublabels, thus helping the feature selection process. SLOFS employs a dynamic sublabel graph to obtain a high-quality sublabel space and uses regularization to constrain label correlations on dynamic sublabel projections. Finally, an effective convergence provable optimization scheme is proposed to solve the SLOFS method. The experimental studies on the 18 datasets demonstrate that the presented method performs consistently better than previous feature selection methods.

Extensive pre-trained visual-language alignment models, such as Contrastive Language-Image Pre-training (CLIP), have demonstrated significant potential for learning representations transferable to domain generation tasks. In hyperspectral image (HSI) classification, a major challenge in deploying such models lies in prompt engineering, which requires particular expertise and substantial time investment. Moreover, existing methods ignore correlation information cross spectral bands. To address these issues, a novel method named learnable prompts-based Transformer (LPFormer) is proposed in this paper. In LPFormer, cross-band correlation information is extracted by self-attention of the transformer, which converted into positional embedding within the transformer framework to obtain the visual features. Subsequently, prompt words are modeled using learnable parameters that turn into efficient expertise. Finally, contrast learning method is used to align visual and textual features. Experimental results on two HSI datasets shows that the proposed LPFormer outperforms other domain adaptation methods.

The oriented target detection algorithm based on deep learning has made significant progress and has been widely applied in various fields, including remote sensing. However, existing methods still face challenges in large-sized targets and targets with similar backgrounds, leading to unsatisfactory detection performance in these scenarios. To address these issues, this paper proposes two modules on the basis of the feature pyramid: the Multi-Scale Feature Fusion Module and the Feature Compensation Module. The Multi-Scale Feature Fusion Module effectively integrates features from different levels, filters out noise introduced during the fusion process, and allows the network to focus more on target regions. The Feature Compensation Module provides semantic information compensation for the highest-level feature map, enhancing the feature representation capability. Extensive experiments were conducted on the DOTA and DIOR-R datasets. The experimental results demonstrate that the introduction of these two modules significantly improves the detection accuracy of the baseline algorithm.

Instance segmentation in the field of remote sensing imagery is recognized as a complex and difficult task. Previous approaches suffer from inadequate feature fusion, insufficient learning of shape information, and lack of segmentation of object edges. To address these challenges, we introduce FEA-Net(Fusion Edge-Aware Instance Segmentation Network), a multiple information fusion model for remote sensing image instance segmentation. Our model makes the predicted instance masks more accurate and can effectively improve the instance segmentation performance of high-resolution remote sensing images. We have evaluated our method on two datasets, NWPU VHR-10, and the iSAID. The experimental results demonstrate the effectiveness of our method, showing strong performance.

Learning object detectors for remote sensing images commonly requires for a huge number of annotated boundary boxes, which are not available without enormous manual efforts in annotating. Alternatively, points can indicate the existence of the objects of interests with reduced labeling cost. Existing Point-supervised object detection (PSOD) methods predominantly employ a two-stage training strategy, which involves propagating point annotations to pseudo boxes at the first stage then training an object detector with these pseudo boxes in a fully supervised manner. However, such paradigm substantially impedes the end-to-end flow of training gradients. In this work, we propose a novel dual-branch network (DBNet) for end-to-end weakly supervised object detection on remote sensing images. Firstly, a pseudo box generation network is attached to the object detector as a sibling branch, which produces semantic response maps for the objects of interest then extracts pseudo boxes by examining their spatial connectivity. Then, instead of training this pseudo box generation network separately, we jointly adjust the pseudo box generation network and the detection network through a multi-task loss. Experimental results on the DOTA-v1.0 dataset demonstrate the effectiveness of our proposed method, achieving an average precision (mAP50) of 32.3%.

The visual question answering (VQA) method applied to remote sensing images (RSIs) can complete the interaction of image information and text information, which avoids professional barriers in different RSIs processing fields. The current methods face challenges in both fully utilizing the global and local information of the image to interact with the question information and addressing the issue of inter-class interference. To address these challenges, this paper proposes a remote sensing visual question answering (RSVQA) model based on the mask-based dual-stream feature mutual attention network (MADNet). First, the dual-stream feature extraction module of the image is used to obtain image features, and the deep and shallow layer feature encoding module is used to obtain question features. Second, the attention mechanism is introduced and combined with the pointwise multiplication method to utilize of the dual-stream features that were extracted in the earlier step. Finally, an answer relevance modulation module based on a binary mask vector is implemented to filter out irrelevant answers. In the experiments, the performance of the proposed strategy is evaluated using two datasets collected by aerial and Sentinel-2 sensors. In our study, we propose a model that outperforms previous approaches, achieving a 6.89% increase in overall accuracy (OA) over the baseline. This enhancement is notable for its persistence, even when the training data is reduced by half, as evidenced by our experiments on the low-resolution dataset.

Rotated object detection is an important research content in the field of remote-sensing images. However, in the rotated object detection, the inconsistency between the loss function and the final detection metric has become an important factor restricting the improvement of detection accuracy. So, in this letter, an ellipse intersection over union (IoU) loss (EPIoU loss) is proposed to solve these problems. The EPIoU loss uses IoU between the bounding boxes’ inscribed ellipses, which is approximate to the original bounding box IoU. This loss function can jointly optimize the prediction box parameters and promote the model to locate the object better. Compared to the complex intersection of rotated rectangles, the intersection calculation of two rotated ellipses is simple. A unified and differentiable process is also designed to calculate EPIoU, which avoids the complexity of the original bounding box IoU calculation. The experiments on DOTA, DIOR, and HRSC datasets verify that the proposed loss function can effectively improve the accuracy of the model.

Significant progress has been achieved in the development of oriented target detection algorithms based on deep learning, which have found widespread application in various fields, including remote sensing. However, existing methods struggle with adjusting model size and often exhibit unsatisfactory detection performance for targets that overlap, are large, or have similar backgrounds. To address these challenges, this letter proposes an oriented target detection algorithm called Oriented you only look once X (YOLOX), which integrates several optimization techniques. Specifically, to meet the requirements of oriented detection while enhancing feature extraction, we introduce a new network architecture that includes an orientation detection branch and a multiscale feature fusion module (MSFFM). An MSFFM based on attention weights is proposed to integrate features across scales while minimizing noise. In addition, to mitigate the impact of the number of positive samples on the original loss function and focus the network’s attention on learning challenging targets, an object-aware reweighted loss function is introduced in this study. This approach dynamically adjusts the loss contribution for each target. Two models of different sizes are developed using the Oriented YOLOX scaling strategy to cater to scenarios prioritizing either accuracy or speed. Extensive experiments on the dataset for object detection in aerial images (DOTA) and object detection in optical remote sensing images (DIOR-R) datasets demonstrate that Oriented YOLOX performs better in detecting challenging targets. Compared with other oriented target detection methods, this approach not only achieves higher detection accuracy but also reduces parameter counts, improving inference speed.

Deep learning methods have gradually developed into the mainstream methods of moving vehicle detection in satellite videos. However, these methods require labor-intensive and time-consuming box-level annotations to predict accurate locations and sizes, which is challenging for large-scale satellite video datasets with hundreds of vehicles. To address this problem, a novel coarse-to-fine dynamic refinement framework (CFDRM) is proposed for moving vehicle detection in satellite videos only under the supervision of point-level annotations. CFDRM generates initial proposal boxes and performs spatio-guided matching with point annotations to obtain coarse box-level pseudo annotations. The initial priority of these coarse annotations is calculated by leveraging locally-consistent prior tailored to satellite videos. Then, a dynamic refinement detector is constructed to transfer coarse annotations to fine annotations with prior and predictive collaborative curriculum refinement. During the curriculum learning process, the coarse annotations are sequentially learned with a certain priority, where the priority is inferred by considering the prior knowledge from the locally-consistent prior and the knowledge itself from the predicted detector. Ultimately, a novel ambiguity-aware loss is designed to optimize the dynamic refinement detector from coarse annotations to fine annotations in an adaptively-weighted fashion. Extensive experiments have been conducted on the Jilin-1 and SkySat satellite video datasets demonstrate the superiority of CFDRM.

The task of hyperspectral image (HSI) classification is fundamental and crucial in HSI processing. Currently, domain adaptive methods have become a research hotspot in HSI classification. However, most domain adaptive methods ignore the class alignment in different domains. Additionally, HSIs have the characteristics of category imbalance and complex spatial-spectral distribution, which restricts the adaptation performance in HSIs. To address these problems, a class-aligned and class-balancing generative domain adaptation (CCGDA) method is proposed for HSI classification. The architecture of CCGDA is designed by using the classifier, domain discriminator, sampler and two weight-sharing generators. In the classifier, split-level capsule network is constructed by extracting rich spatial information of shallow layer and spectral features of deep layer with equivariant characteristic. Then, the classifier provides the pseudo label of samples in the target domain. To prevent the generators from mode collapse caused by category imbalance, the sampler is designed. It samples and re-samples the samples of the target domain in an adaptive proportion according to the statistical calculation through confidence and distribution of pseudo labels. Finally, a novel class-aligned domain adversarial loss is defined to jointly optimize the generators and discriminator. It incorporates the class shift adjusting and adaptive sampling for the samples of the target domain to better adapt the discriminant boundary of the classifier to the target domain. Experiments on benchmark HSI datasets verify the superiority of the proposed method for domain adaptive classification.

Superpixel segmentation algorithms are predominently based on simple linear iterative clustering (SLIC), and treat homogeneous and complex regions equally. This can lead to suboptimal segmentation results, especially in complex images with multiple objects. We address this problem by proposing an SAR image segmentation algorithm based on complicated region-sensitive adaptive superpixel generation and hybrid edge correction (RSASGEC). First, a dynamic initialization algorithm for superpixel seeds based on region complexity is designed. Specifically, a new superpixel representation structure for superpixel seeds is constructed by combining superpixel complexity and the number of contained pixels. The algorithm gives priority to regions with high complexity, dynamically selecting the region with the highest complexity for further partitioning. This results in a dense distribution of superpixel seeds in complex regions, and sparse distributions in homogeneous regions with low complexity. Second, an iterative superpixel segmentation process based on an adaptive energy function is proposed. The Lagrange multiplier mathematical strategy is employed to optimize the adaptive energy function within an adjustable search window, resulting in more compact superpixel segmentation. Finally, a label correction method, based on edge mixture model constraints, is proposed for postprocessing. By integrating edge information from the Gaussian edge detector and the Canny algorithm as constraints, this method leverages majority voting and region growth methods to mitigate edge noise and outliers, refining the superpixel labels. The RSASGEC algorithm is verified in experiments, using one simulated image and six real SAR images. The results indicate that RSASGEC outperforms six representative algorithms, achieving more satisfactory segmentation performance.

Hyperspectral image cross-scene classification (HSICC) poses a significant challenge due to distribution variations between source and target domains. Existing unsupervised domain adaptation methods primarily focus on local knowledge transfer, often neglecting the critical semantic information and sample topological structure inherent in hyperspectral images (HSIs). To address these limitations, this article introduces an end-to-end joint adversarial network with semantic and topology fusion (JAN-STF). This network liberates from the constraints of local perception by integrating semantic and topological information into both domain- and class-level adversarial learning processes. First, the network constructs a semantic-guided cross-domain graph structure to obtain cross-domain features. Subsequently, domain-level adversarial learning is conducted using these features to achieve domain-invariant representation with robust transferability. Moreover, to bolster stability in the ensuing class-level adversarial procedure, the network dynamically computes cross-domain category center distance loss utilizing an intra-domain topological semantic attention mechanism, thereby mapping features to proximate spaces. Finally, class-level adversarial learning is performed by leveraging the prediction discrepancy between the local classifier and the topological classifier, thus enhancing the discriminative performance of the domain-invariant representation. Extensive experiments on three broadly utilized HSICC datasets demonstrate JAN-STF’s superiority in accuracy and Kappa coefficient (KC) metrics over nine leading algorithms.

Multitask learning has been widely applied in visual learning to significantly enhance the performance. The combination of hyperspectral change detection (HCD) and band reweighting can achieve discriminative feature enhancement for improving detection performance. However, existing multitask models for these two tasks are unidirectional, with band reweighting unable to learn from task guidance. To address this challenge, a multitask HCD (MHCD) framework with differential band reweighting and unbalanced contrastive learning is proposed. MHCD consists of a differential band reweighting network (DBRN) and a Siamese detection network. DBRN extracts discriminative information for HCD by analyzing the differential spatial-spectral information across time states, whose optimization is under the guidance of HCD. Furthermore, a multitemporal interaction module and multidomain fusion module are inserted into the Siamese detection network. They hierarchically connect cross-temporal features and fuse features from spatial, spectral, and temporal domains, providing complementary clues in these different domains. Considering the sample imbalance and enormous variation within a class in binary HCD, an unbalanced contrastive learning method based on multiple prototypes (UCLM) tailored has been considered. It estimates multiple prototypes to flexibly adjust the contribution of different classes of samples to the loss. The proposed method has been validated using three public benchmark datasets, demonstrating improvements in multiple metrics for change detection. The code of our paper is available at:https://github.com/jiefeng0109/MHCD.

Unsupervised cross-domain scene segmentation approach adapts the source model to the target domain, which utilizes two-stage strategies to minimize the inter-domain and intra-domain gap. However, the accumulation of errors in the previous stages affects the training of the subsequent stages. In this paper, a framework called statistical and structural domain adaptation (SSDA) is proposed to optimize inter-domain and intra-domain adaptation jointly. Firstly, the statistical inter-domain adaptation (StaIA) is proposed to model dynamic subdomains, which continuously adjust seed samples during the process of domain adaptation to mitigate error accumulation. The dynamic subdomains are modeled by exploring Bayesian uncertainty statistics and global balance statistics, which alleviate the imbalance problem in uncertainty estimation. StaIA encourages the model to transfer comprehensive and genuine knowledge through the seed loss for inter-domain adaptation. Secondly, the structural intra-domain adaptation (StrIA) is proposed to align the intra-domain gap among dynamic subdomains by the structural priors. Specifically, the StrIA models structural priors by truncated conditional random field (TruCRF) loss within the neighborhood, which constrains intra-domain semantic consistency to reduce the intra-domain gap. Experimental results demonstrate the effectiveness of the proposed cross-domain scene segmentation approaches on two commonly-used unsupervised domain adaptation benchmarks. The code is available at https://github.com/ChicalH/SSDA.

Partial multi-label learning (PML) is defined as the construction of robust multi-label classification models from a training set where all instances are correlated with a corresponding group of candidate labels that are only partially accurate. Existing PML approaches have attempted to elicit reliable labels by parsing the guideline information of candidate labels. However, the other side information in the labels that describes what the sample does not contain is largely ignored. Moreover, existing PML approaches only focus on distinguishing the noise information and lack the effective use of noise information. To this end, a partial multi-label learning disambiguation approach guided by negative labels and noise information is proposed. Specifically, a negative label information-inducing paradigm is established based on the constructed negative label encoding matrix. Meanwhile, the negative correlation information guides an iterative label propagation process to induce ground-truth labels with high credibility. In addition, the truth and noise labels are formalized in a unified framework by constructing a regularizer. Moreover, the multi-label predictor is induced by discriminating regularization and disambiguation of label-specific features using the identified noise feature information. Extensive experiments on existing and constructed datasets have demonstrated that the negative label information bootstrapping strategy can be more effective in finding truth labels hidden in candidate labels. Moreover, noisy feature information-induced multi-label prediction outperforms state-of-the-art approaches.

Domain generalization (DG) is one of the critical issues for deep learning in unknown domains. How to effectively represent domain-invariant context (DIC) is a difficult problem that DG needs to solve. Transformers have shown the potential to learn generalized features, since the powerful ability to learn global context. In this article, a novel method named patch diversity Transformer (PDTrans) is proposed to improve the DG for scene segmentation by learning global multidomain semantic relations. Specifically, patch photometric perturbation (PPP) is proposed to improve the representation of multidomain in the global context information, which helps the Transformer learn the relationship between multiple domains. Besides, patch statistics perturbation (PSP) is proposed to model the feature statistics of patches under different domain shifts, which enables the model to encode domain-invariant semantic features and improve generalization. PPP and PSP can help to diversify the source domain at the patch level and feature level. PDTrans learns context across diverse patches and takes advantage of self-attention to improve DG. Extensive experiments demonstrate the tremendous performance advantages of the PDTrans over state-of-the-art DG methods.

The differentiable neural architecture search (NAS) framework has obtained extensive attention and achieved remarkable performance due to its search efficiency. However, most existing differentiable NAS methods still suffer from issues of model collapse, degenerated search-evaluation correlation, and inefficient hardware deployment, which causes the searched architectures to be suboptimal in accuracy and cannot meet different computation resource constraints (e.g., FLOPs and latency). In this article, we propose a novel resource-constrained NAS (ReCNAS) method, which can efficiently search high-performance architectures that satisfy the given constraints, and deal with the issues observed in previous differentiable NAS methods from three aspects: search space, search strategy, and resource adaptability. First, we introduce an elastic densely connected layerwise search space, which decouples the architecture depth representation from the search of candidate operations to alleviate the aggregation of skip connections and architecture redundancies. Second, a scheme of group annealing and progressive pruning is proposed to improve the efficiency and bridge the search-evaluation gap, which steadily forces the architecture parameters close to binary distribution and progressively prunes the inferior operations. Third, we present a novel resource-constrained architecture generation method, which prunes the redundant channel throughout the search based on dynamic programming, making the searched architecture scalable to different devices and requirements. Extensive experimental results demonstrate the efficiency and search stability of our ReCNAS, which is capable of discovering high-performance architectures on different datasets and tasks, surpassing other NAS methods, while tightly meeting the target resource constraints without any tuning required. Besides, the searched architectures show strong generalizability to other complex vision tasks.

When a top-down method is taken to the task of human pose estimation, the accuracy of joint point localization is often limited by the accuracy of human detection. In addition, conventional algorithms commonly encode the image to generate a heat map before processing, but the systematic error in decoding the heat map back to the original image has an impact on the positioning. Therefore, to address the two problems, we propose an algorithm that uses multiple angle models to generate the human boxes and then performs lightweight decoding to recover the image. The new boxes can better fit humans and the recovery error can be reduced. First, we split the backbone network into three sub-networks, the first sub-network is responsible for generating the original human box, the second sub-network is responsible for generating a coarse pose estimation in the boxes, and the third sub-network is responsible for a high-precision pose estimation. In order to make the human box fit the human body better, with only a small number of interfering pixels inside the box, models of the human boxes with multiple rotation angles are generated. The results from the second sub-network are used to select the best human box. Using this human box as input to the third sub-network can significantly improve the accuracy of the pose estimation. Then to reduce the errors arising from image decoding, we propose a lightweight unbiased decoding strategy that differs from traditional methods by combining multiple possible offsets to select the direction and size of the final offset. On the MPII dataset and the COCO dataset, we compare the proposed algorithm with 11 state-of-the-art algorithms. The experimental results show that the algorithm achieves a large improvement in accuracy for a wide range of image sizes and different metrics.

Existing fusion-based local community detection algorithms have achieved good results. However, when assigning a node to a community, similarity functions are sometimes used, which only use node information, while ignoring connection information within the community. These algorithms sometimes fail to find influential nodes, which eventually leads to the failure to find a complete local community. To address these problems, a new local community detection algorithm is proposed in this article. Two strategies, of strong fusion followed by weak fusion, are used alternately to fuse nodes. Compared with using two fusion strategies alone, the alternating loop method can improve the solution of the algorithm in each stage. In strong fusion, we propose a new membership function that considers both node information and connection information in the local community. This improves the quality of the fused node while preserving the structure of the current community. In weak fusion, we propose a parameter-based similarity measure, which can detect influential nodes for a local community. We also propose a local community evaluation metric, which does not require true division to determine the optimal local community under different parameters. Experiments, compared to six state-of-the-art algorithms, show that the proposed algorithm improves accuracy and stability, and also demonstrate the effectiveness of the new local community evaluation metrics in parameter selection.

Band selection is an effective method to deal with the difficulties in image transmission, storage, and processing caused by redundant and noisy bands in hyperspectral images (HSIs). Existing band selection methods usually need to learn a specific model for each HSI dataset, which ignores the inherent correlation and common knowledge among different band selection tasks. Meanwhile, these methods lead to a huge waste of computation. In this article, a novel zero-shot band selection method, called MR-Selection, is proposed for HSI classification. It formalizes zero-shot band selection as a metalearning problem, where advantage actor–critic algorithm-based reinforcement learning (A2C-RL) is designed to extract the metaknowledge in the band selection tasks of various seen hyperspectral datasets through a shared agent. To learn a consistent representation among different tasks, a dynamic structure-aware graph convolutional network is constructed to build a shared agent in A2C-RL. In A2C-RL, the state is tailored in a feasible way and easy to adapt to various tasks. Meanwhile, the reward is defined according to an efficient evaluation network, which can evaluate each state effectively without any fine-tuning. Furthermore, a two-stage optimization strategy is designed to coordinate optimization directions of a shared agent from different tasks effectively. Once the shared agent is optimized, it can be directly applied to unseen HSI band selection tasks without any available samples. Experimental results demonstrate the effectiveness and efficiency of the MR-Selection on the band selection of unseen HSI datasets.

In the last decade, generative adversarial network (GAN) and its variants provide a powerful training mechanism for hyperspectral image (HSI) classification. In HSIs, the distribution of samples is more complicated due to the existence of abundant spatial-spectral information and multi-scale information. The single generation pattern of GANs is prone to modal collapse for the sample generation of HSIs. Moreover, the promotion of the generator only relies on adversarial learning with the discriminator, which limits the generator’s performance. To address these problems, a multi-complementary GANs with contrastive learning (CMC-GAN) is proposed. CMC-GAN consists of two groups of GANs, where coarse-grained GAN adopts the structure in encoder-decoder form for hidden fine-scale and coarse-scale generation, and another fine-grained GAN is responsible for fine-scale generation. In fine-grained GAN, the discriminator is constructed to distinguish the fine-scale samples from different generators, which enforces the joint optimization of these two groups of GANs and makes GANs generate diverse multi-scale samples. Furthermore, a novel contrastive learning constraint is added into GANs, where a unidirectional contrastive loss guarantees the generators to extract intra-class invariant representation and a class-specific contrastive loss urges the discriminators to learn more discriminative features for classification. Finally, both discriminators are adaptively-fused to extract complementary multi-scale spatial-spectral features for classification under the guidance of diverse generated samples. The experimental results demonstrate CMC-GAN has superior classification performance, especially for small sample classification.

The convolutional neural network has achieved remarkable results in most medical image seg- mentation applications. However, the intrinsic locality of convolution operation has limitations in modeling the long-range dependency. Although the Transformer designed for sequence-to-sequence global prediction was born to solve this problem, it may lead to limited positioning capability due to insufficient low-level detail features. Moreover, low-level features have rich fine-grained information, which greatly impacts edge segmentation decisions of different organs. However, a simple CNN module is difficult to capture the edge information in fine-grained features, and the computational power and memory consumed in processing high-resolution 3D features are costly. This paper proposes an encoder-decoder network that effectively combines edge perception and Transformer structure to segment medical images accurately, called EPT-Net. Under this framework, this paper proposes a Dual Position Transformer to enhance the 3D spatial positioning ability effectively. In addition, as low-level features contain detailed information, we conduct an Edge Weight Guidance module to extract edge information by minimizing the edge information function without adding network parameters. Furthermore, we verified the effectiveness of the proposed method on three datasets, including SegTHOR 2019, Multi-Atlas Labeling Beyond the Cranial Vault and the re-labeled KiTS19 dataset called KiTS19-M by us. The experimental results show that EPT-Net has significantly improved compared with the state-of-the-art medical image segmentation method.

In the information age of big data, and increasingly large and complex networks, there is a growing challenge of understanding how best to restrain the spread of harmful information, for example, a computer virus. Establishing models of propagation and node immunity are important parts of this problem. In this article, a dynamic node immune model, based on the community structure and threshold (NICT), is proposed. First, a network model is established, which regards nodes carrying harmful information as new nodes in the network. The method of establishing the edge between the new node and the original node can be changed according to the needs of different networks. The propagation probability between nodes is determined by using community structure information and a similarity function between nodes. Second, an improved immune gain, based on the propagation probability of the community structure and node similarity, is proposed. The improved immune gain value is calculated for neighbors of the infected node at each time step, and the node is immunized according to the hand-coded parameter: immune threshold. This can effectively prevent invalid or insufficient immunization at each time step. Finally, an evaluation index, considering both the number of immune nodes and the number of infected nodes at each time step, is proposed. The immune effect of nodes can be evaluated more effectively. The results of network immunization experiments, on eight real networks, suggest that the proposed method can deliver better network immunization than several other well-known methods from the literature.

Band selection is an important step in efficient processing of hyperspectral images (HSIs), which can be seen as the combination of powerful band search technique and effective evaluation criterion. The existing deep-learning-based methods make the network parameters sparse to search the spectral bands using threshold-based functions or regularization terms. These methods may lead to an intractable optimization problem. Furthermore, these methods need to repeatedly train deep networks for evaluating candidate band subsets. In this article, we formalize hyperspectral band selection as a reinforcement learning (RL) problem. Band search is regarded as a sequential decision-making process, where each state in the search space is a feasible band subset. To evaluate each state, a semisupervised convolutional neural network (CNN), called EvaluateNet, is constructed by adding the intraclass compactness constraint of both limited labeled and sufficient unlabeled samples. A simple stochastic band sampling method is designed to train EvaluateNet, making it possible to efficiently evaluate without any fine-tuning. In RL, new reward functions are defined by taking the EvaluateNet and the penalty of repeated selection into account. Finally, advantage actor–critic algorithms are designed to explore in the state space and select the band subset according to the expected accumulated reward. The experimental results on HSI data sets demonstrate the effectiveness and efficiency of the proposed algorithms for hyperspectral band selection.

Generative adversarial network (GAN) has been rapidly developed because of its powerful generating ability. However, imbalanced class distribution of hyperspectral images (HSIs) easily causes mode collapse in GAN. Moreover, limited training samples in HSIs restrict the generating ability of GAN. These issues may further deteriorate the classification performance of the discriminator. To conquer these issues, a novel self-supervised divide-and-conquer GAN (SDC-GAN) is proposed for HSI classification. In SDC-GAN, a pretext cluster task with an encoder-decoder architecture is designed by leveraging abundant unlabeled samples. By transferring the learned cluster representation from the cluster task, limited labeled samples are divided effectively in the downstream classification. According to the division of clustering, SDC-GAN constructs a generic and several specific branches for both the generator and discriminator. The generator generates all-class and specific-class samples by using the generic and specific branches separately and combines them adaptively. It can weaken the generation preference for the classes with large sample sizes and alleviate the mode collapse problem. Meanwhile, the classification ability of the discriminator is improved by integrating the judgment of specific branches into the generic branch. Experimental results show that SDC-GAN achieves competitive results for HSI classification compared with several state-of-the-art methods.

Semantic segmentation is an important yet unsolved problem in aerial scenes understanding. One of the major challenges is the intense variations of scenes and object scales. In this paper, we propose a novel multi-scale aware-relation network (MANet) to tackle this problem in remote sensing. Inspired by the process of human perception of multi-scale information, we explore discriminative and diverse multi-scale representations. For discriminative multi-scale representations, we propose an inter-class and intra-class region refinement method (IIRR) to reduce feature redundancy caused by fusion. IIRR utilizes the refinement maps with intra- and inter-class scale variation to guide multi-scale fine-grained features. Then, we propose multi-scale collaborative learning (MCL) to enhance the diversity of multi-scale feature representations. The MCL constrains the diversity of multi-scale feature network parameters to obtain diverse information. And the segmentation results are rectified according to the dispersion of the multi-level network predictions. In this way, MANet can learn multi-scale features by collaboratively exploiting the correlation among different scales. Extensive experiments on image and video datasets which have large scale variations have demonstrated the effectiveness of our proposed MANet.

Recently, most convolutional neural network-based methods use convolutional kernels of fixed size to extract features, which ignore the inherent spatial structure information of ground objects and lose spatial details. In addition, rough first-order statistics is not enough to capture subtle differences between different categories and extract non local context information. To address these issues, a hyperspectral image (HSI) classification method based on multi-scale cross-branch response and second-order channel attention (MCRSCA) is proposed in this paper. Firstly, a multi-scale cross-branch response module (MCBR) is proposed, which uses convolution kernels of different sizes for feature extraction. It adds and concatenates the features of different scales respectively to obtain rich and complementary spatial context information. Then, element multiplication and element addition are performed on the fused multi-scale features to promote the propagation of the multi-scale information and enhance the nonlinear expression ability. Next, the second-order channel attention module (SOCA) is designed to interact the channel information through the feature covariance matrix to obtain the long-term dependence between channels. This module pays more attention to the significant channels and suppresses the redundant channels. Finally, the residual connection is used to embed MCBR and SOCA into the residual block to improve the gradient back propagation and accelerate the training process. Experiments on four commonly used HSI benchmark datasets show that the results of MCRSCA is competitive compared with other state-of-the-art methods.

This paper proposes a novel segmentation algorithm for synthetic aperture radar (SAR) images. The algorithm performs region-level segmentation based on edge feature and label assistance (REFLA). It demonstrates improved performance in terms of segmentation accuracy while better preserving image edges. Firstly, an edge detection scheme is implemented, which fuses information from two advanced edge detection methods, thereby obtaining a more precise edge strength map (ESM). Secondly, a Canny algorithm is performed to divide the SAR image into edge regions and homogeneous regions, and different smoothing templates are selected according to pixel positions. Therefore, an anisotropic smoothing on the SAR image can be achieved, aiming at suppressing the noise within targets while also accurately maintaining the target boundaries. Thirdly, K-means clustering is applied on the smoothed result, to generate an initial set of labels. Using ESM and the initial labels as inputs, a watershed transformation and a majority voting strategy are employed to realize an initial segmentation at the region level. Finally, a label-aided region merging (LaRM) strategy is used to correctly segment the wrongly labeled regions, to give the final segmentation result. The LaRM, with merging rules based on label rather than gray characteristics, can avoid the need for calculating a large number of complex formulae, thus accelerating the region merging. Results are presented of experiments, on both simulated and real SAR images, in which the proposed REFLA method is compared against six state-of-the-art algorithms from the literature. REFLA achieves higher accuracy, while better retaining the image edges.

Synthetic aperture radar (SAR) is widely used in the field of modern remote sensing due to its high resolution for a comparatively small antenna. However, there are still some difficulties in the processing of SAR images. In particular, accurate segmentation of small targets and image corners remains an important challenge, as these can easily be lost during conventional image smoothing and denoising methods. To address this, we propose an SAR image segmentation algorithm based on constrained smoothing and hierarchical label correction (CSHLC). First, a Canny algorithm is used to extract the edges of SAR images, and the Gaussian smoothing is performed on SAR images under edge constraints to achieve noise reduction so that the edges of small and big targets are well preserved. Second, a preliminary K-means clustering is conducted on the smoothing results, and then, a Markov random field (MRF) model is used on the clustering results (“original label” results), iteratively calculating a maximum likelihood set of pixel labels. Finally, through two label correction methods, pixel group counting comparison (PGCC) and gray similarity comparison (GSC), the labels of the MRF output are further checked and corrected to obtain final segmentation results. Compared with seven state-of-the-art algorithms, simulation results on both simulated SAR images and real SAR images show that the proposed CSHLC delivers higher accuracy while better retaining corners and small targets.

Nonlocal convolutional neural networks have difficulties in dealing with the imbalanced samples in hyperspectral images effectively, so the networks cannot achieve ideal experiment results. Therefore, this paper proposes an adaptive projection attention-based simplified nonlocal neural network for hyperspectral image classification. Firstly, the local information is calculated in horizontal and vertical directions. Then the information is passed to a simplified nonlocal network to learn the global semantic information. The simplified nonlocal network can reduce information redundancy and improve classification accuracy at the same time. Secondly, the global semantic information is adaptively projected according to the spatial features and compressed using multi-scale pooling layers. After that, the pooled results are reassigned channel weights through two fully connected layers and extended using multi-scale pooling layers. Then the extended features are concatenated with the global semantic information, which can alleviate the imbalanced sample existing in the dataset. Then a simplified nonlocal approach is used to fuse shallow and deep information to improve the robustness and classification performance of the network. In this paper, experiments of the proposed method are conducted on three widely used hyperspectral datasets compared with those of seven state-of-the-art algorithms, and satisfactory overall and average accuracies are achieved, demonstrating the effectiveness of the proposed algorithm.

Attention mechanism-based Hyperspectral Image (HSI) classification algorithms typically extract spectral and spatial features by spectral attention and spatial attention network respectively. However, these algorithms lack joint attention and ignore imbalanced samples, leading to insufficient information extraction. To address this problem, this paper proposes a novel HSI classification algorithm based on the pyramidal coordinate attention and weighted self-distillation (PCA-WSD). To perform the joint attention of spectral and spatial features, the proposed PCA mechanism uses spectral attention to cope with the diverse spatial features. The PCA mechanism consists of two components. First, the spatial pyramid coordinate squeeze (SPCS) is designed to aggregate spatial features with local and global information. Then, the tailored spatial pyramid coordinate excitation (SPCE) adaptively enhances their informative spectral features for the obtained spatial features, realizing the joint attention to spectral-spatial features. Further, considering the imbalance of samples, WSD is proposed. Specifically, weighted cross-entropy is integrated into WSD. Extensive experiments are evaluated on the four HSI benchmark datasets: Indian Pine (IP), Pavia University (UP), Kennedy Space Center (KSC), and Pavia Center (PC). Compared with the seven advanced algorithms, experimental results of the proposed algorithm1. reveal superior classification performance, especially for the imbalanced samples.

Deep learning methods have gained rapid development in hyperspectral pansharpening (HP) due to powerful spatial–spectral feature extraction ability. However, most of these methods are optimized using a single reconstruction objective. It is difficult for these methods to find a balance between spectral preservation and spatial preservation. Furthermore, these methods adopt interpolation or convolution to upsample the hyperspectral images (HSIs), which tends to cause noticeable spectral distortion. To conquer these issues, a novel multiobjective guided divide-and-conquer network (MO-DCN) is proposed for HP. It consists of a deconvolution long short-term memories (LSTMs) network (DLSTM) and a divide-and-conquer network (DCN). DLSTM leverages bi-direction learning to upsample HSIs by considering 3-D spatiotemporal dependencies. Then, DCN designs a two-branch architecture to reconstruct spatial and spectral information from upsampled HSIs and panchromatic images (PANIs), respectively, where the spatial branch designs an attention-in-attention module (AIAM) to emphasize complementary attention in a coarse-to-fine way. Finally, co-improvement of spatial and spectral information is formulated as an Epsilon-constraint-based multiobjective optimization. The Epsilon constraint method transforms one objective into a constraint and regards it as a penalty bound to make an excellent tradeoff between different objectives. Experimental results demonstrated that the proposed method markedly improves pansharpening performance in both the spatial and spectral domains and has superior fusion performance than state-of-the-art methods.

Band selection has been widely utilized in hyperspectral image (HSI) classification to reduce the dimensionality of HSIs. Recently, deep-learning-based band selection has become of great interest. However, existing deep-learning-based methods usually implement band selection and classification in isolation, or evaluate selected spectral bands by training the deep network repeatedly, which may lead to the loss of discriminative bands and increased computational cost. In this article, a novel convolutional neural network (CNN) based on bandwise-independent convolution and hard thresholding (BHCNN) is proposed, which combines band selection, feature extraction, and classification into an end-to-end trainable network. In BHCNN, a band selection layer is constructed by designing bandwise 1×1 convolutions, which perform for each spectral band of input HSIs independently. Then, hard thresholding is utilized to constrain the weights of convolution kernels with unselected spectral bands to zero. In this case, these weights are difficult to update. To optimize these weights, the straight-through estimator (STE) is devised by approximating the gradient. Furthermore, a novel coarse-to-fine loss calculated by full and selected spectral bands is defined to improve the interpretability of STE. In the subsequent layers of BHCNN, multiscale 3-D dilated convolutions are constructed to extract joint spatial-spectral features from HSIs with selected spectral bands. The experimental results on several HSI datasets demonstrate that the proposed method uses selected spectral bands to achieve more encouraging classification performance than current state-of-the-art band selection methods.

Convolutional neural networks (CNNs) have demonstrated outstanding performance on image classification. To classify the hyperspectral images (HSIs), existing CNN-based approaches commonly adopt the architecture using single or several fixed spatial windows as inputs. This kind of architecture may lose contextual information or incorporate heterogeneous information due to the neglect of various land-cover distributions in HSIs. To deal with this problem, a novel attention multibranch CNN method based on adaptive region search (RS-AMCNN) is proposed for HSI classification. In RS-AMCNN, sizes and locations of spatial windows are searched in the nonlocal candidate region adaptively according to sample-specific distribution. These flexible spatial windows are input into several branches of RS-AMCNN. In each branch, convolutional long short-term memories (ConvLSTMs) are merged into CNN from shallow to deep layers, which not only extracts joint spatial-spectral features, but also exploits complementary information among different layers. Then, a branch attention mechanism is devised to emphasize more discriminative branches and suppress less useful ones. It forces RS-AMCNN to extract multiscale and multicontextual attention features for classification. Finally, RS-AMCNN is optimized end-to-end by combining the losses from the ramose classifiers of different branches and the main classifier. Experiments carried on several benchmark HSI data sets demonstrate that RS-AMCNN provides promising classification performance, especially in edge preservation and region uniformity.

As a fundamental but challenging task in the interpretation of remote sensing images, scene classification plays an important role in various applications and has become an active research topic. Many previous works have demonstrated the remarkable performance of the deep convolutional neural networks (CNNs) for remote sensing scene classification. However, the progress made by CNN-based methods for scene classification has gradually reached saturation in recent years, due to the serious dependence on the pretrained CNN models, the limitations of manually designed network architecture and the disadvantages of existing data sets. In this article, a new paradigm to automatically design a suitable CNN architecture for scene classification is investigated. We propose an efficient architecture search framework to discover optimal network architectures in continuous search space with the gradient-based optimization method. Our framework consists of two stages: the search phase and the evaluation phase. During the search process, a greedy and progressive search strategy is introduced to search network building blocks (i.e., cells) through bilevel optimization. Besides, we propose a simple architecture regularization scheme to further improve the search efficiency and the robustness of discovered architectures. After the search process, the optimal cell architectures are determined and then repeatedly stacked to construct the final network for evaluation. For the data set, we propose a mergence strategy to build a new large-scale remote sensing scene image data set that contains rich scene categories and image diversity, making it feasible to find a new CNN model with strong generalization ability for scene classification. Extensive experiments demonstrate the efficiency of the proposed search strategies and the impressive classification performance of searched CNN architectures on seven public benchmark data sets, including four large-scale data sets and three small-scale data sets.

Semi-supervised non-negative matrix factorization (NMF) exploits the strengths of NMF in effectively learning local information contained in data and is also able to achieve effective learning when only a small fraction of data is labeled. NMF is particularly useful for dimensionality reduction of high-dimensional data. However, the mapping between the low-dimensional representation, learned by semi-supervised NMF, and the original high-dimensional data contains complex hierarchical and structural information, which is hard to extract by using only single-layer clustering methods. Therefore, in this article, we propose a new deep learning method, called semi-supervised graph regularized deep NMF with bi-orthogonal constraints (SGDNMF). SGDNMF learns a representation from the hidden layers of a deep network for clustering, which contains varied and unknown attributes. Bi-orthogonal constraints on two factor matrices are introduced into our SGDNMF model, which can make the solution unique and improve clustering performance. This improves the effect of dimensionality reduction because it only requires a small fraction of data to be labeled. In addition, SGDNMF incorporates dual-hypergraph Laplacian regularization, which can reinforce high-order relationships in both data and feature spaces and fully retain the intrinsic geometric structure of the original data. This article presents the details of the SGDNMF algorithm, including the objective function and the iterative updating rules. Empirical experiments on four different data sets demonstrate state-of-the-art performance of SGDNMF in comparison with six other prominent algorithms.

In order to reduce the influence of noise and obtain better change detection effect, this paper proposes a method for SAR image change detection based on mean shift pre-classification and fuzzy C-means. First, the original image is pre-classified based on mean shift clustering. As a clustering method with non-parametric density estimation, mean shift can effectively maintain the edge information of the object, and can smooth the pixel intensity of the same type of object to reduce the influence of noise on change detection. Then, the difference map is generated by the log-ratio operator and classified into changed area, uncertain area, and unchanged area. After the adjustment, the pre-classification is performed by mean shift and the difference map is generated. Finally, the improved FCM algorithm is used to classify the difference map to generate change detection result map. The effectiveness of the proposed method is verified by experiments with different contrast algorithms on real SAR image datasets.

With the rapid development of various technologies of satellite sensor, synthetic aperture radar (SAR) image has been an import source of data in the application of change detection. In this paper, a novel method based on a convolutional neural network (CNN) for SAR image change detection is proposed. The main idea of our method is to generate the classification results directly from the original two SAR images through a CNN without any preprocessing operations, which also eliminate the process of generating the difference image (DI), thus reducing the influence of the DI on the final classification result. In CNN, the spatial characteristics of the raw image can be extracted and captured by automatic learning and the results with stronger robustness can be obtained. The basic idea of the proposed method includes three steps: it first produces false labels through unsupervised spatial fuzzy clustering. Then we train the CNN through proper samples that are selected from the samples with false labels. Finally, the final detection results are obtained by the trained convolutional network. Although training the convolutional network is a supervised learning fashion, the whole process of the algorithm is an unsupervised process without priori knowledge. The theoretical analysis and experimental results demonstrate the validity, robustness, and potential of our algorithm in simulated and real data sets. In addition, we try to apply our algorithm to the change detection of heterogeneous images, which also achieves satisfactory results.

This paper puts forward a new method for SAR image change detection based on conditional spatial and kernel fuzzy C-means (CSKFCM). In CSKFCM algorithm, first, the kernel function is introduced into FCM, and the improved FCM is used to cluster the difference image to obtain the membership matrix. The kernel function maps the dataset to high dimensional space by nonlinear mapping, the same time, it can avoid the curse of dimensionality. Meanwhile, the kernel distance replaces the non-robust Euclidean distance of FCM algorithm, which can compensate the defect that FCM is sensitive to speckle noise to some extent. Second, this paper uses conditional spatial method to modify the membership matrix, and obtain the new membership matrix and cluster center. This method uses the neighbor and spatial information of pixel to modify the membership matrix again, which can obtain more accurate membership matrix. In this paper, the principle analysis and experiments show that the proposed algorithm can suppress speckle noise better than the contrast algorithm, and can get more accurate change detection results.

Feature selection is an important approach for reducing the dimension of high-dimensional data. In recent years, many feature selection algorithms have been proposed, but most of them only exploit information from the data space. They often neglect useful information contained in the feature space, and do not make full use of the characteristics of the data. To overcome this problem, this paper proposes a new unsupervised feature selection algorithm, called non-negative spectral learning and sparse regression-based dual-graph regularized feature selection (NSSRD). NSSRD is based on the feature selection framework of joint embedding learning and sparse regression, but extends this framework by introducing the feature graph. By using low dimensional embedding learning in both data space and feature space, NSSRD simultaneously exploits the geometric information of both spaces. Second, the algorithm uses non-negative constraints to constrain the low-dimensional embedding matrix of both feature space and data space, ensuring that the elements in the matrix are non-negative. Third, NSSRD unifies the embedding matrix of the feature space and the sparse transformation matrix. To ensure the sparsity of the feature array, the sparse transformation matrix is constrained using the -norm. Thus feature selection can obtain accurate discriminative information from these matrices. Finally, NSSRD uses an iterative and alternative updating rule to optimize the objective function, enabling it to select the representative features more quickly and efficiently. This paper explains the objective function, the iterative updating rules and a proof of convergence. Experimental results show that NSSRD is significantly more effective than several other feature selection algorithms from the literature, on a variety of test data.

Nonnegative matrix factorization (NMF) is well known to be an effective tool for dimensionality reduction in problems involving big data. For this reason, it frequently appears in many areas of scientific and engineering literature. This letter proposes a novel semisupervised NMF algorithm for overcoming a variety of problems associated with NMF algorithms, including poor use of prior information, negative impact on manifold structure of the sparse constraint, and inaccurate graph construction. Our proposed algorithm, nonnegative matrix factorization with rank regularization and hard constraint (NMFRC), incorporates label information into data representation as a hard constraint, which makes full use of prior information. NMFRC also measures pairwise similarity according to geodesic distance rather than Euclidean distance. This results in more accurate measurement of pairwise relationships, resulting in more effective manifold information. Furthermore, NMFRC adopts rank constraint instead of norm constraints for regularization to balance the sparseness and smoothness of data. In this way, the new data representation is more representative and has better interpretability. Experiments on real data sets suggest that NMFRC outperforms four other state-of-the-art algorithms in terms of clustering accuracy.

In fuzzy clustering algorithm, fuzzy possibilistic C-means clustering algorithm (FPCM) is widely used. However, the method is sensitive to its parameters and the clustering accuracy and robustness is poor. In order to overcome the above problems, this paper presents an intuitionistic fuzzy possibilistic C-means clustering based on genetic algorithm (IFPCM-GA). IFPCM-GA does not only retain the advantages of FPCM, but also uses a kernel function to replace the Euclidean distance to enhance the robustness of the algorithm. We get an intuitionistic fuzzy possibilistic C-means clustering algorithm (IFPCM) by using the intuitionistic fuzzy set theory to the fuzzy possibilistic C-means clustering algorithm induced by kernel metric. Taking into account the hesitation degree of data, IFPCM can obtain a more accurate membership matrix and cluster centers to enhance the clustering performance. IFPCM-GA uses genetic algorithm to search the optimal parameters of IFPCM, which can avoid the poor clustering results. The experimental results show that IFPCM-GA has a strong robustness and can obtain more accurate clustering results compared with the existing algorithms.

Cognitive radio is a new network technology developed in recent years, which focuses on the low utility of spectrum in the wireless communication system. This paper proposes a new algorithm based on the immune clone optimization for unconstrained multi-objective resource allocation in the downlink OFDMA network. We first convert the constraint of data transmission rate proportionality of each user to a proportional objective function, which avoids dealing with the proportion constraint. And an effective coding method is adopted to deal with constraints in subcarrier allocation. Compared with the traditional method, this method can get less code length, which narrows the solution space significantly. Moreover, we use variable multiples clone operation and heuristic mutation operation based on some priori information. Finally, the population is updated in two steps: first, remove some antibodies with high value of proportional objective function; second, choose Pareto dominant solution from the remaining individuals to update the population. Simulation results show that the proposed algorithm can get better performance compared to the previous works.

With the application of community detection in complex networks becoming more and more extensive, the application of more and more algorithms for community detection are proposed and improved. Among these algorithms, the label propagation algorithm is simple, easy to perform and its time complexity is linear, but it has a strong randomness. Small communities in the label propagation process are easy to be swallowed. Therefore, this paper proposes a method to improve the partition results of label propagation algorithm based on the pre-partition by circularly searching core nodes and assigning label for nodes according to similarity of nodes. First, the degree of each node of the network is calculated. We go through the whole network to find the nodes with the maximal degrees in the neighbors as the core nodes. Next, we assign the core nodes’ labels to their neighbors according to the similarity between them, which can reduce the randomness of the label propagation algorithm. Then, we arrange the nodes whose labels had not been changed as the new network and find the new core nodes. After that, we update the labels of neighbor nodes according to the similarity between them again until the end of the iteration, to complete the pre-partition. The approach of circularly searching for core nodes increases the diversity of the network partition and prevents the smaller potential communities being swallowed in the process of partition. Then, we implement the label propagation algorithm on the whole network after the pre-partition. Finally, we adopt a modified method based on the degree of membership determined by the bidirectional attraction of nodes and their neighbor communities. This method can reduce the possibility of the error in partition of few nodes. Experiments on artificial and real networks show that the proposed algorithm can accurately divide the network and get higher degree of modularity compared with five existing algorithms.

Spectral-spatial representation based algorithms have been widely applied in hyperspectral image (HSI) classification, which exploit the fact that pixels in a local patch often have similar spectral reflectance values and probably belong to the same class. Collaborative representation (CR) is a typical supervised classification method for high-dimensional data, which has been widely used for spectral-spatial representation based HSI classification. However, it suffers from the degraded representation of redundant and irrelative pixels when all of the labeled pixels are used as a dictionary for representation. In this letter, a novel method, local CR with adaptive dictionary selection, is proposed to solve this problem, in which we first average the values of pixels from local patches to incorporate the contextual information of neighbors, and then, an adaptive dictionary selection method is presented to select the most similar pixels to each test pixel from the dictionary to reduce the influence of redundant and irrelevant pixels in representation. Experimental results on two HSIs show that the proposed method outperforms some spectral-spatial representation based algorithms in terms of classification accuracy.

The capacitated arc routing problem (CARP) has attracted considerable attention from researchers due to its broad potential for social applications. This paper builds on, and develops beyond, the cooperative coevolutionary algorithm based on route distance grouping (RDG-MAENS), recently proposed by Mei et al. Although Mei's method has proved superior to previous algorithms, we discuss several remaining drawbacks and propose solutions to overcome them. First, although RDG is used in searching for potential better solutions, the solution generated from the decomposed problem at each generation is not the best one, and the best solution found so far is not used for solving the current generation. Second, to determine which sub-population the individual belongs to simply according to the distance can lead to an imbalance in the number of the individuals among different sub-populations and the allocation of resources. Third, the method of Mei et al. was only used to solve single-objective CARP. To overcome the above issues, this paper proposes improving RDG-MAENS by updating the solutions immediately and applying them to solve the current solution through areas shared, and then according to the magnitude of the vector of the route direction, and a fast and simple allocation scheme is proposed to determine which decomposed problem the route belongs to. Finally, we combine the improved algorithm with an improved decomposition-based memetic algorithm to solve the multiobjective large scale CARP (LSCARP). Experimental results suggest that the proposed improved algorithm can achieve better results on both single-objective LSCARP and multiobjective LSCARP.

This paper proposes a quantum immune clone algorithm to solve the constrained multi-objective optimization problem. Firstly, constraints deviation value is added to objective function value to form a new objective function value, which translates the constrained multi-objective optimization problem into an unconstrained multi-objective optimization problem. Secondly, it does not only retain the feasible non-dominated solutions, but also utilizes the non-feasible solutions which have small constraint deviation value and objective function value. The appearing of the non-feasible solutions expands the search scope and makes it easy to evolve solutions near the Pareto front. Then, a quantum rotating gate is designed to accelerate the computational speed. At last, crossover and mutation are used to obtain better individuals. Compared with the state-of-art algorithm, simulation results show that the proposed algorithm has a better improvement on GD distance and on the diversity.

There can be a complicated mapping relation between decision variables and objective functions in multi-objective optimization problems (MOPs). It is uncommon that decision variables influence objective functions equally. Decision variables act differently in different objective functions. Hence, often, the mapping relation is unbalanced, which causes some redundancy during the search in a decision space. In response to this scenario, we propose a novel memetic (multi-objective) optimization strategy based on dimension reduction in decision space (DRMOS). DRMOS firstly analyzes the mapping relation between decision variables and objective functions. Then, it reduces the dimension of the search space by dividing the decision space into several subspaces according to the obtained relation. Finally, it improves the population by the memetic local search strategies in these decision subspaces separately. Further, DRMOS has good portability to other multi-objective evolutionary algorithms (MOEAs); that is, it is easily compatible with existing MOEAs. In order to evaluate its performance, we embed DRMOS in several state of the art MOEAs to facilitate our experiments. The results show that DRMOS has the advantage in terms of convergence speed, diversity maintenance, and portability when solving MOPs with an unbalanced mapping relation between decision variables and objective functions.

Many-objective problem has more than 3 objectives. Because of the extraordinary difficulty of acquiring their Pareto optimal solutions directly, traditional methods will be out of operation for such problems. In recent years, many researchers have turned their attention to the study of this area. They are interested in two areas: acquiring some part of Pareto front which is useful to the researchers (Preferred Solutions) and reducing redundant objectives. In this paper, we combine two dimension reduction methods: the method based on Pareto optimal solution analysis and the method based on correlation analysis, to form a novel algorithm for dimension reduction. Firstly, the Pareto optimal solutions are acquired through NSGA-II. Then the objectives who contribute little to the number of non-dominated solutions are removed. At last, the dimension of objectives is reduced further according to their contribution to the principal component in PCA analysis. In this way, we can acquire the right non-redundant objectives with low time complexity. Simulation results show that the proposed algorithm can effectively reduce redundant objectives and keep the non-redundant objectives with low time.

Research on multiobjective optimization (MO) becomes one of the hot points of intelligent computation. Compared with evolutionary algorithm, the artificial immune system used for solving MO problems (MOPs) has shown many good performances in improving the convergence speed and maintaining the diversity of the antibody population. However, the simple clonal selection computation has some difficulties in handling some more complex MOPs. In this paper, the simple clonal selection strategy is improved and a novel immune clonal algorithm (NICA) is proposed. The improvements in NICA are mainly focus on four aspects. 1) Antibodies in the antibody population are divided into dominated ones and nondominated ones, which is suitable for the characteristic of one multiobjective optimization problem has a series Pareto-optimal solutions. 2) The entire cloning is adopted instead of different antibodies having different clonal rate. 3) The clonal selection is based on the Pareto-dominance and one antibody is selected or not depending on whether it is a nondominated one, which is different from the traditional clonal selection manner. 4) The antibody population updating operation after the clonal selection is adopted, which makes antibody population under a certain size and guarantees the convergence of the algorithm. The influences of the main parameters are analyzed empirically. Compared with the existed algorithms, simulation results on MOPs and constrained MOPs show that NICA in most problems is able to And much better spread of solutions and better convergence near the true Pareto-optimal front.

To effectively preserve the edges of a synthetic aperture radar (SAR) image when despeckling, an algorithm with edge detection and fuzzy clustering in the translation-invariant second-generation bandelet transform (TIBT) domain is proposed in this letter. A Canny operator is first utilized to detect and remove edges from the SAR image. Then, TIBT and fuzzy C-mean clustering are employed to decompose and despeckle the edge-removed image, respectively. Finally, the removed edges are added to the reconstructed image. The algorithm suggests each coefficient in high-frequency subbands as the clustering feature, proposes a calculation method of the best clustering number, and defines the signal and noise in the clustering results. Experimental results show that the visual quality and evaluation indexes outperform the other methods with no edge preservation. The proposed algorithm effectively realizes both despeckling and edge preservation and reaches the state-of-the-art performance.

In this study, we introduce a hybrid immune algorithm based on the intelligent recombination operator and clonal selection algorithm. The intelligent recombination operator uses orthogonal experimental design for factor analysis which identifies the potential gene segments from two individuals to improve their antigenic affinities. The new algorithm, termed as Hybrid Immune Algorithm with Recombination (HIAR), can avoid the decrease of gene diversity in evolutionary process. It evaluates the hamming distance before recombination and uses the two individuals which have the largest hamming distance between each other to implement intelligent recombination operator. It is shown empirically that HIAR has better performance in solving benchmark functions as compared with Intelligent Evolutionary Algorithm and Clonal Selection Algorithm.

In this study, we introduce two improved assessment metrics of multiobjective optimizers, Nondominated Ratio and Spacing Distribution, and analyze their rationality and validity. Based on the concept of Immunodominance and Antibody Clonal Selection Theory, a novel multiobjective optimization algorithm, Immune Dominance Clonal Multiobjective Algorithm (IDCMA), is put forward. The simulation comparisons between IDCMA and the Strength Pareto Evolutionary Algorithm show that IDCMA has the best performance in popular metrics such as Spacing, Coverage of Two Sets and the two new metrics presented in this paper when low-dimensional multiobjective problems are concerned. The statistical results of the four metrics also show that Spacing Distribution conquers some limitations of Spacing triumphantly, and Nondominated Ratio conquers the limitation of Coverage of Two Sets that only compared between two sets.