The deployment of federated learning in real-world IoT ecosystems presents intrinsic challenges stemming from hardware asymmetry and sample scarcity, the existing related approaches generally homogenize model architectures and assume abundant labeled data, resulting in an inability to achieve fast generalization on devices with varying computational capabilities and dynamic task conditions. To address the aforementioned challenges, we propose a novel federated cross-device heterogeneous few-shot adaptation (Fed-CHFSA) method for IoT systems. In Fed-CHFSA, collaborating with other devices, each edge device obtains a personalized model that can not only adapt well to the category distribution of respective local data but also recognize unseen categories without data leakage. Specifically, we designed a fine-grained personalized aggregation (FPA) module and an information entropy-driven adaptive feature constraint (EAFC) module for the devices possessing a small amount of labeled data in the model aggregation and training phases of Fed-CHFSA, respectively. In each round of global communication, the edge device performs a certain epoch of personalized training locally under the normalization of EAFC in the feature space. Subsequently, the central server follows the FPA to finely aggregate the received model updates parameter-wise, and redistribute the updated global model to participating devices. After multiple rounds of global communication, every edge device acquires an optimal model more adaptable to local data and more generalized to unseen categories. Compared with existing FL and PFL algorithms on three benchmark few-shot learning (FSL) datasets, the proposed Fed-CHFSA framework achieves the best performance. The effectiveness of FPA and EAFC is also demonstrated by extensive ablation experiments.

Deep learning-based methods have achieved significant success in remote sensing Earth observation data analysis. Numerous feature fusion techniques address multimodal remote sensing image classification by integrating global and local features. However, these techniques often struggle to extract structural and detail features from heterogeneous and redundant multimodal images, particularly in label-scarce scenarios. With the goal of introducing frequency domain learning to model key and sparse detail features, this paper introduces the spatial–spectral-frequency interaction network (S 2 Fin), which integrates pairwise fusion modules across the spatial, spectral, and frequency domains. Specifically, we propose a high-frequency sparse enhancement transformer to refine spectral signatures by adaptively enhancing discriminative high-frequency components. For spatial-frequency interaction, we present a depth-wise strategy: the adaptive frequency channel module fuses low-frequency structural information with enhanced details in shallow layers, while the high-frequency resonance mask amplifies modality-consistent regions in deep layers using phase similarity. In addition, a spatial–spectral attention fusion module bridges the gap between spectral and spatial branches at intermediate depths. Extensive experiments on four benchmark datasets demonstrate that S 2 Fin exhibits good robustness and generalization, and its performance significantly outperforms state-of-the-art methods in few-sample settings. The code is available at https://github.com/HaoLiu-XDU/SSFin .

Cognitive diagnosis (CD) is a fundamental task within computational social systems, essential for personalizing learning in intelligent education by assessing a learner’s fine-grained knowledge proficiency. A critical dilemma exists in designing cognitive diagnosis models (CDMs): conventional psychometric models are interpretable but often simplistic, while complex deep learning models achieve high accuracy at the cost of becoming uninterpretable “opaque models.” This lack of transparency is a major barrier to trust and adoption in real-world educational social systems. To address this challenge, this article introduces probabilistic model-building for cognitive architecture search (PMB-CAS), a novel framework that automates the discovery of CDMs that balance diagnostic accuracy with structural interpretability. We reframe the design process as a multi-objective optimization problem, balancing diagnostic performance [area under the curve (AUC)] with a refined, operator-weighted model interpretability score (MIS) that fairly assesses topological complexity. The framework navigates a flexible, tree-based search space using a probabilistic model-building strategy to efficiently discover promising architectures. Extensive experiments on benchmark datasets demonstrate that PMB-CAS discovers a portfolio of Pareto-optimal models that not only achieve state-of-the-art accuracy, outperforming established baselines, but also offer varying degrees of structural transparency. This work provides educators and stakeholders with a spectrum of trustworthy solutions and establishes a new paradigm for developing effective and verifiable models for computational social systems.

Federated recommendation systems aim to provide high-quality recommendations while protecting user privacy. However, existing federated recommendation algorithms typically use a unified item embedding framework, where all clients share the same item representations. Such approaches often fail to capture users’ personalized perceptions of the same item and struggle to reflect subtle changes in user preferences, limiting the effectiveness of personalized recommendations. Moreover, statistical heterogeneity among clients poses additional challenges for model optimization. To address these issues, we propose APFMRec, a personalized federated recommendation framework that enhances both personalization and global optimization. APFMRec incorporates an adaptive item embedding module to dynamically adjust item representations based on individual user preferences. In addition, a meta-learning update module is designed to mitigate statistical heterogeneity and improve collaborative optimization across clients. Extensive experiments on five real-world datasets demonstrate the effectiveness of APFMRec. The proposed framework consistently outperforms existing federated recommendation methods, achieving up to 8.1% improvement in HR@10 and 7.8% improvement in NDCG@10.

In recent years, adversarial attacks in hyperspectral image (HSI) classification have garnered increasing attention. However, existing attack methods primarily manipulate individual pixel spectral to mislead deep neural networks (DNNs) into misclassification, overlooking the physical consistency of hyperspectral data. This oversight results in adversarial samples that lack physical interpretability and suffer from low attack efficiency. To alleviate these issues, this paper proposes a sparse unmixing guided adversarial attack framework (SUGAA) to efficiently generate hyperspectral adversarial samples that satisfy physical consistency. The proposed framework first employs sparse unmixing to extract the abundance matrix of HSI, introducing adversarial perturbations to the abundance matrix to generate physically consistent adversarial samples. Additionally, SUGAA leverages the compositional similarity of materials within intra-class HSI pixels to design a class-specific perturbation generation strategy, enhancing the applicability of adversarial perturbations across pixels of the same class. To further improve optimization effectiveness, SUGAA incorporates a class-specific perturbation optimization algorithm based on momentum iterative gradients to avoid local optima, ensuring stable and efficient perturbation generation. Experimental results on real HSI datasets demonstrate that SUGAA not only generates adversarial samples with high attack performance and physical consistency but also exhibits robustness to common preprocessing transformations.

Modeling visual perception in a manner consistent with human subjective evaluation has become a central direction in both video quality assessment (VQA) and broader visual understanding tasks. While free-energy-guided self-repair mechanisms—reflecting human observational experience—have proven effective in image quality assessment, extending them to VQA remains non-trivial. In addition, biologically inspired paradigms such as holistic perception, local analysis, and gaze-driven scanning have achieved notable success in high-level vision tasks, yet their potential within the VQA context remains largely underexplored. To address these issues, we propose EyeSimVQA, a novel VQA framework that incorporates free-energy-based self-repair. It adopts a dual-branch architecture, with an aesthetic branch for global perceptual evaluation and a technical branch for fine-grained structural and semantic analysis. Each branch integrates specialized enhancement modules tailored to distinct visual inputs—resized full-frame images and patch-based fragments—to simulate adaptive repair behaviors. We also explore a principled strategy for incorporating high-level visual features without disrupting the original backbone. In addition, we design a biologically inspired prediction head that models sweeping gaze dynamics to better fuse global and local representations for quality prediction. Experiments on five public VQA benchmarks demonstrate that EyeSimVQA achieves competitive or superior performance compared to state-of-the-art methods, while offering improved interpretability through its biologically grounded design. Our code will be publicly available at https://github.com/handsomewzy/EyeSim-VQA.

Various deep learning-based methods have greatly improved hyperspectral image (HSI) classification performance, but these models are sensitive to noisy training labels. Human annotation on remote sensing images inevitably introduced label noise, which degrades the model prediction confidence. Understanding the spatial characteristics and distribution of such annotation errors is crucial for both diagnosing dataset annotation failures and guiding effective robust learning strategies. Current noisy label learning methods pay limited attention to visualizing noise label distributions, and these approaches often exhibit poor compatibility with noise-free models. Leveraging the relationship between the prediction uncertainty and label noise, we propose a Local Bayesian Framework (LBF) for noisy HSI classification and noise labels awareness. LBF adapts standard CNN, GCN, or Transformer backbones via local Bayesian adaptation (LBA) to evaluate prediction uncertainty and employs an uncertainty-monitoring optimization strategy (U-MOS) for training. Without major architectural changes, LBF delivers accurate uncertainty maps that highlight noisy regions, suppresses overfitting to corrupted labels, and consistently improves classification robustness across four benchmark HSI datasets.

Local smoothness is a widely used prior in hyperspectral image (HSI) denoising tasks. The current work mainly realizes the representation of this prior through total variation (TV) regularization. However, the TV regularization applies a uniform penalty to each entry in the image, unable to effectively balance noise removal and texture preservation. Aiming at this problem: 1) We propose a novel regularization for HSI denoising called Spatial-Spectral Texture-Preserved Total Variation (SSTPTV). This naturally expresses the physical phenomenon of the difference in sparsity between textured regions and smooth regions. Specifically, the regularization relaxes the sparsity penalty in textured regions by a weight learning strategy and sparsity measurement method for gradient maps, thereby preserving the spatial-spectral textures of HSIs. 2) An HSI denoising model based on the SSTPTV regularization constraint is given. We propose a tensor alternating subspace representation method that can capture the overall spatial texture features across all bands and the overall spectral texture features across all spectral curves. By applying the SSTPTV regularization constraint to these subspaces, the spatial-spectral texture structures are effectively preserved. An efficient ADMM-based algorithm for solving the model is designed. The simulated and real noise removal experiments of HSI prove that the proposed method has significant superiority and can serve as a framework to optimize other TV-based denoising methods. The code is available at https://github.com/zth-code/SSTPTV.

Binary neural networks (BNNs) have been applied in limited resources and mobile devices because of their extreme model compression ability. However, manually designing suitable architectures is challenging given the specialized structure of binarized operations. Neural architecture search (NAS) provides a promising approach for designing high-performance BNN architectures. In practice, various situations require networks with different parameter sizes and performance levels. Therefore, this article proposes a multiobjective evolutionary NAS algorithm for BNNs based on a two-stage training strategy (MO-TS-BNAS) to solve these problems. First, the ApproxSign function is used to approximate the gradient error in the training of BNNs. To avoid the small model trap problem, two auxiliary objectives are introduced in nondominated sorting to retain larger models with similar errors. Then, a two-stage training strategy with flexible use of auxiliary objectives is proposed, forming the selection mechanism in environmental selection. The path dropout method is used in the second stage to prevent hypernetwork overfitting. In addition, the mini-batch gradient descent strategy is improved to speed up individual architecture evaluation and reduce time cost in the search process. Finally, the full precision baseline search space is binarized for general experimental comparison. Our MO-TS-BNAS algorithm balances the two different objective functions of the model size and error. A large number of experiments are carried out on the CIFAR10 and ImageNet datasets, and the results show the effectiveness of the proposed method.

Large language models (LLMs) have demonstrated remarkable capabilities across various natural language processing tasks, but they remain vulnerable to adversarial attacks and pose significant security concerns. Existing attack methods often treat adversarial prompts as flat sequences, neglecting the rich hierarchical structure of natural language, which could limit their effectiveness. Advancing the methodologies for adversarial attacks is crucial for rigorously assessing the security of LLMs and identifying subtle vulnerabilities. This paper introduces AdvGP, a novel framework that leverages genetic programming (GP) to generate adversarial prompts for LLMs. AdvGP exploits the inherent structural similarities between GP trees and natural language syntax to optimize the structure of harmful prompts. The framework incorporates a multi-granularity hierarchical attack strategy, specialized genetic operators that leverage an assisting LLM for depth-aware crossover and multi-level mutation, and a comprehensive fitness function integrating semantic consistency and attack effectiveness. The proposed method achieves competitive attack performance on multiple LLMs, consistently generating harmful outputs despite higher perplexity than some baselines. Ablation studies confirm the significant contributions of both LLM-aided and depth-aware mechanisms to AdvGP’s effectiveness. Furthermore, transferability analysis reveals that the generated prompts are able to bypass the defenses of various state-of-the-art LLMs, such as ChatGPT and Gemini.

For expensive multiobjective optimization problems, there exists useful knowledge, e.g., the trained surrogate models, can be transferred to assist the optimization of a target optimization problem, which is termed as multi-problem surrogates. Stacking transfer is able to combine the pretrained source surrogate models and the preliminary target model with a meta-regression algorithm to transfer knowledge from source to target. However, when large-scale source models are involved in the many-problem scenarios, the less correlated sources may hurt the target performance, which is known as negative transfer. In this paper, sparse representation of the coefficients of meta-regression is considered to automatically select the most relevant source models for largely avoiding negative transfer. In the proposed many-problem surrogates, the coefficients of the source and target models are assumed to be sparse under the non-negativity and sum-to-one constraints. Then a sparse transfer stacking model is established with l1-norm of the coefficients. Next, the alternating direction method of multipliers is employed to solve the resulting constrained optimization problem by converting it into several much simpler problems. Most of the previous works assume that the costs for evaluation have no much difference and this assumption rarely holds in the real-world applications. In order to further reduce the total costs, an improved surrogate model with a cost-sensitive measure is designed to estimate the cost and select new solutions for real evaluation based on their estimated fitness, uncertainty and cost. Experimental results on synthetic and practical problems have demonstrated the superiority of the proposed many-problem surrogates.

Data-driven evolutionary algorithms (DDEAs) have achieved significant success in numerous real-world optimization problems, where exact objective functions and constraint functions do not exist, and they mainly rely on available data. However, the existing DDEAs primarily focus on improving performance through data and surrogate, without considering that the users may lack the specialized domain knowledge and sufficient computing resources required for DDEAs. To address the aforementioned issues, this paper proposes a novel paradigm called Evolutionary Learning and Optimization as a Service (ELOaaS) and investigates the potential collusion attacks between machine learning modules and evolutionary computing modules on cloud server, which may lead to privacy leakage. Consequently, a privacy-enhanced DDEA (PEDDEA) is proposed as an instantiation algorithm of ELOaaS, which is designed to tackle offline data-driven evolutionary optimization within the ELOaaS paradigm. In the proposed PEDDEA, a subspace learning-based privacy protection strategy is designed to defense the collusion attacks. Additionally, a model management strategy based on Kendall tau metric is introduced to construct high-quality surrogate ensembles. PEDDEA enables users to outsource private offline data to cloud servers, thereby approaching the optimal solution while ensuring privacy protection. Comprehensive experiments are conducted on benchmark problems and safety evaluation problems of autonomous vehicles. According to the experimental results, the proposed algorithm has significant performance advantages over existing offline DDEAs while ensuring privacy protection.

In the realm of real-world knowledge graphs, the dynamism of facts is a prevailing characteristic. To illustrate, a popular restaurant was awarded a Michelin star in 2004 and retained this prestigious recognition in 2008, but lost it in 2012 due to changes in management and menu quality. This sequence highlights how neglecting temporal context can lead to misconceptions about factual accuracy. Furthermore, the relations intertwining distinct entities or the same entity across different chronological markers exhibit complexity and hierarchy. Regrettably, existing methods for temporal knowledge graph relation prediction fall short in following challenges: they lack a nuanced, hierarchical comprehension of knowledge structure and fail to adeptly integrate temporal dynamics with static attributes. Addressing these issues, this study introduces Hyperbolic-based Temporal Knowledge Graph Relation Prediction (HTKGP) approach to harness the power of hyperbolic geometry. Our innovation is an attention-guided, learnable curvature mechanism designed to preserve and enrich the intricate semantic hierarchy inherent in data. Besides, we propose a longitudinal information entity embedding strategy due to the plentiful temporal information. This not only captures the enduring impact of past events on present states but also achieves efficiency through parameter reduction. Empirical validation across multiple datasets shows HTKGP efficiently navigates the rich semantic landscape within hyperbolic spaces and yields superior predictive performance. Our implementations are publicly available at: https://github.com/jianruichen/HTKGP .

3D Change Detection (3DCD) has gradually become another research hotspot after image change detection. Recent works focus on using artificial labels for supervised or weakly-supervised training of siamese networks to segment changed points. However, labeling every points of multi-temporal point clouds is very expensive and time-consuming. In addition, these works lack effective self-supervised signals, and existing self-supervised signals often fail to capture sufficiently rich change information. To solve this problem, we assume that the powerful representation of 3D objects should model the consistency information of unchanged regions and distinguish different objects. Based on this assumption, we propose a new unsupervised framework called MUCD to learn change information of multi-temporal point clouds through bidirectional optimization of change segmentor and feature extractor. The training of network is divided into two stages. We first design a foreknowledge point contrastive loss based on the characteristics of the 3DCD task to initialize the feature extractor, and then propose a masked consistency loss to further learn the shared geometric information of unchanged regions in the multi-temporal point clouds, utilizing it as a free and powerful supervised signal to train a change segmentor. In the inference stage, only the segmentor is used to take multi-temporal point clouds as input and produce change segmentation result. Extensive experiments are conducted on SLPCCD and Urb3DCD, two real-world datasets of streets and urban buildings, to verify that our proposed unsupervised method is highly competitive and even outperforms supervised methods in scenes where semantic information changes occur, exhibiting better performance in generalization ability and robustness.

When the current physical adversarial patches cannot deceive thermal infrared detectors, the existing techniques implement adversarial attacks from scratch, such as digital patch generation, material production, and physical deployment. Besides, it is difficult to finely regulate infrared radiation. To address these issues, this paper designs an adversarial thermal display (AdvDisplay ) by assembling thermoelectric coolers (TECs) as an array. Specifically, to reduce the gap between patches in the physical and digital worlds and decrease the power of AdvDisplay device, heat transfer loss and electric power loss are designed to guide the patch optimization. In addition, a precise temperature control scheme for AdvDisplay is proposed based on proportional-integral-derivative (PID) control. Due to the accurate temperature regulation and the reusability of AdvDisplay , our method is able to improve the attack success rate and the efficiency of physical deployments. Extensive experimental results indicate that the proposed method possesses superior adversarial effectiveness compared to other methods and demonstrates strong robustness in physical attacks.

Federated few-shot learning (FedFSL) aims to enable the clients to obtain personalized generalization models for unseen categories with only a small number of referenceable samples in the distributed collaborative training paradigm. Most existing FedFSL-related algorithms suffer from domain bias and feature coupling in the presence of data heterogeneity and sample scarcity. In this work, we propose a collaborative feature representation disentanglement (CFRD) scheme for FedFSL to address these issues. After each client receives the global aggregation parameters, the original feature representation is decoupled into global communal features and local personality features with personalized bias representation, to maintain both global consistency and local relevance in the first feature representation disentanglement. On the few-shot metric space about the second feature representation disentanglement, category-independent information is encoded by class-specific and class-irrelevant reconstructions to separate the discriminative features. The proposed scheme collaboratively accomplishes global domain bias feature disentanglement and local category degradation feature disentanglement from client-wise and class-wise. Experiments on three few-shot benchmark datasets conforming to the FedFSL paradigm demonstrate that our proposed method outperforms state-of-the-art approaches in both global generality and local specificity.

We propose a transformation diffusion model for point cloud registration to balance precision and efficiency. Our method formulates point cloud registration as a denoising diffusion process from noisy transformation to object transformation, which is represented by quaternion and translation. Specifically, in training stage, object transformation diffuses from ground-truth transformation to random distribution, and the model learns to reverse this noising process. In sampling stage, the model refines randomly generated transformation to the optimal transformation in a progressive way. We derive the variational bound in closed form for training and provide instantiation of the model. Our diffusion model maps transformation into latent space, and splits the transformation into two components (rotation and translation) based on the fact that they belong to different solution spaces. In addition, our work provides the following crucial findings: (i) Point cloud registration, one of the representative discriminative tasks, can be solved by a generative way and mapped into latent space to obtain new unified probabilistic formulation. (ii) Our model, Transformation Diffusion Model (TDM) can be a plug-and-play agent for point cloud registration, making our method applicable to different deep registration networks. Experimental results on synthetic and real-world datasets demonstrate that, in correspondence-free and correspondence-based scenarios, TDM can both achieve exceeding 60% performance improvements and higher efficiency simultaneously.

Learning representations from numerous 2D image data has shown promising performance, yet very few works apply this representations to point cloud registration. In this paper, we explore how to leverage the 2D information to assist the point cloud registration, and propose IAPReg, an Image-Assisted Partial 3D point cloud Registration framework with the multi-view images generated by the input point cloud. It is expected to enrich 3D information with 2D knowledge, and leverage 2D knowledge to assist with point cloud registration. Specifically, we create multi-view depth maps by projecting the input point cloud from several specific views, and then extract 2D and 3D features using some well-established models. To fuse the information learned from 2D and 3D modalities, inter-modality multi-view learning module is proposed to enhance geometric information and complement semantic information. Weighted SVD is a common method to reduce the impact of inaccurate correspondences on registration. However, determining the correspondence weights is not trivial. Therefore, we design a 2D-weighted SVD method, where the 2D knowledge is employed to provide weight information of correspondences. Extensive experiments perform that our method outperform the state-of-the-art method without additional 2D training data.

Federated cross-domain recommendation aims to alleviate the problem of data sparsity and enable collaborative modeling of user behavior data from different platforms or institutions while ensuring data privacy. Most existing federated cross-domain recommendation methods rely on item IDs for modeling, ignoring the mining and utilization of item semantic information. In addition, due to the heterogeneity of data between different domains, the model is prone to domain bias and feature coupling problems during the aggregation process, which negatively impacts the recommendation performance. This paper proposes a dual-target federated cross-domain recommendation framework with semantic enhancement and disentangled contrastive learning. First, to utilize semantic information of items, item IDs features and text semantic features are jointly fused to enhance the item embedding representations. Second, we propose a user representation decoupling mechanism to explicitly decouple users preferences into shared and domain-specific preferences, thereby alleviating domain bias and feature coupling problems. Furthermore, we design a cross-domain contrastive learning module on the server side to enhance the consistency and transferability of shared representations between user representations across different domains. Experimental results show that the proposed algorithm performs significantly better than existing optimal methods on multiple real-world datasets, demonstrating its excellent performance in federated cross-domain recommendations.

Human pose generation is a complex task due to the non-rigid and highly variable nature of human body structures and appearances. However, existing methods often overlook the fundamental differences between spatial transformations of poses and texture generation for appearance, which makes them prone to overfitting. To address this issue, we propose a multi-pose generation framework driven by disentangled pose and appearance guidance. Our approach includes a Global-aware Pose Generation module that iteratively generates pose embeddings, enabling effective control over non-rigid body deformations. Additionally, we introduce the Global-aware Transformer Decoder, which leverages similarity queries and attention mechanisms to achieve spatial transformations and enhance pose consistency through a Global-aware block. In the appearance generation phase, we condition a diffusion model on pose embeddings produced in the initial stage and introduce an Appearance Adapter that extracts high-level contextual semantic information from multi-scale features, enabling further refinement of pose appearance textures and providing appearance guidance. Extensive experiments on the UBC Fashion and TikTok datasets demonstrate that our framework achieves state-of-the-art results in both quality and fidelity, establishing it as a powerful approach for complex pose generation tasks.

Point cloud registration is a fundamental task in 3D computer vision. Recent advances have shown that graph-based methods are effective for outlier rejection in this context. However, existing clique-based methods impose overly strict constraints and are NP-hard, making it difficult to achieve both robustness and efficiency. While the k-core reduces computational complexity, which only considers node degree and ignores higher-order topological structures such as triangles, limiting its effectiveness in complex scenarios. To overcome these limitations, we introduce the $k$-truss from graph theory into point cloud registration, leveraging triangle support as a constraint for inlier selection. We further propose a consensus voting-based low-scale sampling strategy to efficiently extract the structural skeleton of the point cloud prior to $k$-truss decomposition. Additionally, we design a spatial distribution score that balances coverage and uniformity of inliers, preventing selections that concentrate on sparse local clusters. Extensive experiments on KITTI, 3DMatch, and 3DLoMatch demonstrate that our method consistently outperforms both traditional and learning-based approaches in various indoor and outdoor scenarios, achieving state-of-the-art results.

In the field of Internet of Things (IoT), data distillation has been thought of as a key method to condense the original real dataset into a tiny synthetic dataset with less training burden while maintaining as much data utility as possible for training deep learning models. However, the data synthesis process may remember some sensitive information about the original dataset, which may raise privacy concerns for data owners. To address this problem, we present a novel differential privacy (DP)-based data distillation algorithm. Specifically, in the data distillation phase, we first randomly pick a training model from the model pool in each epoch, and then build a fine-grained distribution matching to generate informative data for improving the task-oriented model performance. In the privacy preservation phase, we selectively perturb input features that are more important for model training based on the sparse vector technique to protect the sensitive information contained in the original dataset and reduce privacy costs. Extensive experiments across several real-world datasets demonstrate that our algorithm can achieve higher data utility and model accuracy than existing solutions.

Federal learning provides a secure platform for sharing sensitive data, yet imposes stringent requirements on the data. Non-IID data often cannot fully enjoy the convenience it offers. When clients possess divergent feature sets, retaining only the common features is a prevalent yet suboptimal practice. This paper proposes a novel omnidirectional federated learning framework that employs a Star collaboration network designed to leverage independent information from client nodes for feature reconstruction of other clients. It establishes an approximate distribution network, reinforcing feature correlations while overcoming data isolation seen in traditional federal learning. Additionally, homomorphic encryption is utilized to ensure data security throughout the transmission process. Experimental evaluations on structured datasets demonstrate that the reconstructed prediction results closely approximate those under the condition of complete data, confirming the effectiveness of the Star network in data completion and multi-party prediction scenarios.

Hyperspectral change detection is critical for analyzing the temporal evolution of the feature components in multi-temporal hyperspectral images. However, existing methods often fall short of fully exploiting the spatio-temporal-spectral correlations within these images, thereby limiting their accuracy and robustness. This paper introduces a novel hyperspectral change detection method, termed dual collaborative sparse unmixing via variable splitting augmented Lagrangian and total variation (DCLSUnSAL-TV). By integrating dual collaborative sparsity and total variation regularizers, this method capitalizes on the local similarity of changes in the feature components, leveraging the low-rank property of hyperspectral difference images (HSDIs) and their inherent spatial-spectral correlations. A customized abundance-wise-truncation and ensemble strategy is designed to obtain the change map by aggregating the subpixel-level changes with respect to each endmember. Comprehensive comparison and ablation experiments demonstrate the effectiveness of the proposed method in improving the accuracy of change detection. The source code is available at https://github.com/2alsbz/DCLSUnSAL_TV.

3D Single Object Tracking (SOT) plays an important role in real-world visual applications such as autonomous driving and planning. How to realize effective 3D SOT is still a valuable challenge due to its carrier-sparse point clouds and its role-complex influencing factors. Inspired by the remote modeling of popular transformers, we further propose a Versatile Point Tracking Transformer (VPTT) method for 3D SOT, with object guidance from the template point cloud to the search area point cloud under the siamese-based tracking paradigm. Specifically, VPTT employs self- and cross- attention mechanisms and extends four matching operations, resulting in leveraging the contextual information of consecutive frames to improve the tracking results. By constructing a deep network VerFormer consisting of four successive transformer layers, which performs matching operations involving fusional transformation, separative discrimination, intersectional interaction, and unidirectional propagation from shallow to deep. Considering that the tracking task involves multiple processes, VPTT further learns how to forecast intermediate outputs including mask probability, trailing distance, and heading angle at each stage. Such a specialized design allows our VPTT to revisit the end-to-end training paradigm used for 3D tracking while developing a versatile transformer that is a perfect fit for the 3D SOT task. Experiments on three benchmarks, KITTI, nuScenes, and Waymo, show that VPTT achieves state-of-the-art tracking performance on siamese-based tracking running at $\sim$∼62 FPS.

Recommender systems play crucial roles in addressing the issue of information overload, but traditional centralized storage in recommendation poses significant privacy concerns. In recent years, federated learning has been successfully introduced into a recommendation, while these algorithms still encounter several challenges. First, real-world recommendation scenarios often suffer from sparse data, making it difficult for models to learn reliable representations. Second, data heterogeneity necessitates the design of personalized models to enhance recommendation performance. To address these challenges, we propose a federated recommendation approach based on graph neural networks, named federated personalized contrastive learning for recommendation. On the client side, we propose a contrastive learning approach to enhance the embedding quality of nodes (users or items) by maximizing positive similarities. Specifically, we formulate the concept of structural neighbors based on the graph structure and devise a contrastive learning objective. We treat nodes and their structural neighbors as positive pairs to better learn node representations. On the server side, we group users based on the learned representations and compute cluster-level federated models and a global model. Each user learns a personalized model by combining these two models. Extensive experiments on five real-world datasets demonstrate that the proposed algorithm outperforms existing methods in terms of performance.

Existing 3D mask learning methods encounter performance bottlenecks under limited data, and our objective is to overcome this limitation. In this paper, we introduce a triple point masking scheme, named TPM, which serves as a scalable plug-and-play framework for MAE pre-training to achieve multi-mask learning for 3D point clouds. Specifically, we augment the baseline methods with two additional mask choices (i.e., medium mask and low mask) as our core insight is that the recovery process of an object can manifest in diverse ways. Previous high-masking schemes focus on capturing the global representation information but lack fine-grained recovery capabilities, so that the generated pre-training weights tend to play a limited role in the fine-tuning process. With the support of the proposed TPM, current methods can exhibit more flexible and accurate completion capabilities, enabling the potential autoencoder in the pre-training stage to consider multiple representations of a single 3D point cloud object. In addition, during the fine-tuning stage, an SVM-guided weight selection module is proposed to fill the encoder parameters for downstream networks with the optimal weight, maximizing linear accuracy and facilitating the acquisition of intricate representations for new objects. Extensive experimental results and theoretical analysis show that five baselines equipped with the proposed TPM achieve comprehensive performance improvements on various downstream tasks. Our code and models are available athttps://github.com/liujia99/TPM.

Research on adversarial attacks in remote sensing tasks have predominantly focused on designing perturbations or patches, presenting challenges in balancing attack success rate and adversarial stealthiness. Instead of focusing on designing adversarial examples under adversarial perturbation constraints to ensure stealthiness, this paper proposes a Style Transfer Enabled Adversarial Attack with Attention Mechanism (STEAM), which leverages style transfer to generate adversarial examples with high visual fidelity. Specifically, STEAM transfers distinctive styles from critical regions of source samples to attackable areas in target samples, effectively incorporating natural textures from source samples. To further refine this process, an attention mechanism is introduced to selectively extract style features from key regions of the source samples, mitigating redundancy from global style information. Additionally, selective style transfer process also includes the consideration of semantic features across different regions in target samples, ensuring a more effective attack area selection. As a result, STEAM achieves high attack success rate by utilizing proper style selected from groups of style samples, and preserving high visual fidelity through selectively transfer the natural style feature into specific attackable region in target samples. Experimental results on the UCM and WHU-RS19 datasets demonstrate that STEAM not only enhances the visual fidelity of adversarial examples but also improves the attack success rate. Furthermore, experiments against state-of-the-art adversarial defense methods highlight the adversarial attack effectiveness and robostness of STEAM compared to other adversarial attack methods.

Deep learning methods excel in Polarimetric SAR (PolSAR) image classification. However, existing methods typically sample an image block for each pixel with a fixed-size square window, which always contains inconsistent/incomplete content with the central pixel, resulting in many misclassifications especially in boundary and heterogeneous regions. So, a size-fixed square window is not enough for representing various terrain objects. To address this issue, we develop a content-adaptive multi-region deep network to obtain contextual consistent sampling windows for diverse terrain objects. Firstly, a complex scene of PolSAR image is partitioned into homogeneous, heterogeneous and boundary regions. Then, sampling windows with adaptive direction and scale are designed for three distinct regions. Besides, windows with central and global regions are proposed to provide additional local and global information. Finally, a fusion network is designed to adaptively combine different sampling windows to enhance classification performance. Experimental results on three real data sets demonstrate that the proposed method can achieve superior performance in both edge details and heterogeneous terrain objects compared with the state-of-the-art methods.

The unstructured, unordered and inherent irregular sampling properties presents difficulties for accurate and efficient realizing semantic segmentation of large-scale 3D point cloud. The complexity and the long-distance information exploitation are the key challenges for large-scale 3D point cloud semantic segmentation. Therefore, in order to efficiently exploit long-distance global information and improve the segmentation accuracy of point cloud data located at the edges of distinct categories, a novel Trigonometric Bilinear attention and Global-aware Aggregation network (TBGA-Net) is designed to integrate and supplement the local-global contextual features for large-scale point clouds segmentation. The proposed Global-aware Context Aggregation block (GCA) can implicitly excavate the global context for each 3D point by utilizing its surface-to-volume ratio of the neighborhood to the global point cloud. Furthermore, aim at refining local-global features, the Trigonometric Bilinear Attention block (TBA) utilizes trigonometric functions to embed the point cloud coordinates of local regions, and applies bilinear attention to realize feature enhancement for obtaining more discriminative local-global features. Additionally, we further designed a novel Dynamic-adjusting Cross-entropy Loss (DCLoss) to incorporate with TBGA-Net for addressing the issue of class imbalance in training data for large-scale 3D point cloud semantic segmentation. The experimental results on three 3D point cloud datasets demonstrates that the proposed algorithm indicates better segmentation accuracy especially for the point located at the boundary of the distinct categories.

Unsupervised point cloud registration is crucial in 3D computer vision. However, most unsupervised methods struggle to construct effective optimization objectives and reliable unsupervised signals to enhance the performance of the model. To address these issues, with the observation of the significant alignment between the registration process and the Markov Decision Process (MDP), we model point cloud registration as MDP, which can provide more reliable unsupervised signals through the reward. We propose a colored noise based cross-entropy method, which introduces colored noise into sampling process, regulating the power spectral density of the action sequence and expanding the search space, improving the registration effect. Particularly, to strengthen constraints on MDP and training in the transformation space, we utilize equivariance theory to construct transformation equivariant constraint as a new optimization objective and derive equivariant constraint solutions for optimization, providing more reliable unsupervised signals. Extensive experiments demonstrate the superior performance of our method on benchmark datasets.

Anomaly detection in surveillance videos aims to differentiate anomalies from regular events by discriminative representations, which has gathered considerable attention due to its significant effect to public security. However, most existing works are limited in the lack of annotated samples, and lots of approaches find it challenging to avoid the well-reconstruction of anomalous data. To alleviate these issues, we propose a dual distillation fusion framework for weakly supervised anomaly detection. We reformulate the anomaly detection problem into two steps, namely filtering anomalies and inpainting normal patterns. Each step corresponds to one branch of the dual distillation. Specifically, the dual distillation comprises the contrastive distillation module and the inpainting distillation module. The contrastive distillation optimizes the encoder to filter out abnormal features and capture key normal features, while the inpainting distillation refines the decoder to inpaint normal patterns on the encoded features. The contrastive distillation module and the inpainting distillation module are optimized iteratively in a self-training manner with video-level labeled data. Moreover, a joint optimization module is devised to effectively fuse the distilled encoder and decoder, thereby collectively improving the anomaly detection performance. During the training phase, we take into account the diversity of normal samples by selecting pseudo normal and abnormal samples with high confidence from abnormal videos. These selected samples, along with original normal frames, are then fed into the subsequent training iterations to enhance the distinguishing ability of the model. Experimental results show that our proposed method performs competitively on five benchmark datasets.

Constrained by imaging systems, hyperspectral images (HSIs) always have a low spatial resolution. Deep learning-based HSI super-resolution methods have achieved impressive results through learning the nonlinear mapping between low-resolution (LR) and high-resolution (HR) images. However, most of them take the LR image or its upsampled version through bicubic interpolation as input, leading to low-quality features and limited details captured by the network. As a powerful generative model, diffusion model has the ability to learn both contextual semantics and textual details from distinct timesteps, enabling the effective exploration of spatial-spectral distributions in high-dimensional data. In this paper, we propose a novel method that extracts high-quality prior information from original images to assist in super-resolution through pretraining a diffusion model. Specifically, we first train a diffusion model using original HSI patches in a self-supervised manner and then obtain prior features from the pretrained denoising U-Net decoder. To efficiently incorporate the prior features into the super-resolution model, we propose an adaptive fusion module based on spatial and spectral attention mechanisms, which enhances features in both dimensions while preserving the original characteristics. Additionally, to leverage the complementarity of spatial and spectral information, we design a spatial-spectral aggregation Transformer module that incorporates an adaptive interaction module to facilitate information exchange across different dimensions, thereby enhancing the representation capability. Extensive experiments on three public hyperspectral datasets demonstrate that the proposed method achieves excellent super-resolution performance and outperforms the state-of-the-art methods in terms of quantitative quality and visual results.

Transfer evolutionary multiobjective optimization leverages the relevant knowledge from other source problems (distinct but possibly related) to assist the optimization of the target problem of interest. Multi-problem surrogates stack multiple source surrogates to reduce the number of function evaluations of the target expensive problem. The current multi-problem surrogates only considers several source problems and the source and target problems are assumed to be homogeneous. In order to address the above issues, this paper proposes fast heterogeneous multi-problem surrogates for transfer evolutionary multiobjective optimization with a large number of surrogates. First, an iterative surrogate selection strategy is designed to select the highly relevant surrogates from the large-scale surrogate pool to avoid negative transfer. Second, heterogeneous multi-problem surrogates are established to align the features of the source and target models. Finally, an adaptive k-fold cross-validation method is proposed to obtain the predicted values of the target model with low computational costs. Experiments on the multiobjective optimization benchmark problems and multiobjective neural architecture search problems have demonstrated that the proposed method is able to avoid negative transfer in the large-scale scenarios and reduce the computational costs.

3D point cloud registration is a process of solving the geometric transformation between two point clouds. This process is an important issue in computer vision and pattern recognition. The registration methods based on geometric features are highly sensitive to the scale of feature extraction. Changes in scale can introduce inaccuracies in feature descriptions, thereby compromising the reliability of the registration results. To mitigate the impact of feature scale on the outcomes and the high-dimensional issue arising from features of different scales, we propose a method for multi-scale point cloud feature selection. We solve the high-dimensional problem of feature selection by designing a multi-task framework. By designing a mutual information dimensionality reduction method, we decomposed the high-dimensional feature selection task of different descriptors with multi-scale features into multiple related low-dimensional feature selection tasks. Then, by means of the knowledge transfer among these low-dimensional feature selection tasks, we sought the best feature subset to obtain more robust feature information. We evaluate the effectiveness of our method by conducting extensive experiments on various datasets. The experimental results show that the method outperforms other feature descriptors in terms of descriptive power and robustness and improves the effectiveness of point cloud registration.

Point cloud registration (PCR) is an important task for other point cloud tasks. Feature-based methods are widely adopted for their speed and efficiency in PCR. The descriptive capability of features extracted by a single geometric descriptor is limited. Descriptive capabilities can be improved by concatenating features extracted from multiple descriptors. However, due to the existence of redundant and irrelevant features, the correct corresponding points are difficult to match, which further affects the registration effect. We propose an evolutionary multitasking point cloud descriptor optimization method. Integrate existing descriptors to optimize descriptors with stronger description ability. Labeling features to calculate the feature importance for the registration and generating multitasks. In optimized processing, approximate evaluation which is calculated by prior correspondence saved in the database replaces the expensive searching correspondences process in the entire point cloud. Finally, a multiscale filter is developed to remove error correspondences by the geometric information from multiple scale descriptor features. Experimental demonstrate that the proposed approach can optimize a feature subset with higher-descriptive capability compared to other methods and show superior PCR performance on 14 point cloud models. This is the first paper on point cloud descriptor optimization, which provides a new idea for PCR research.

Deep neural networks (DNNs) have achieved remarkable progress in remote sensing image (RSI) analysis, yet their vulnerability to subtle adversarial perturbations poses a critical threat to safety-critical applications such as environmental monitoring. While black-box attacks have garnered attention for their practicality, existing methods face a dilemma in RSI scenarios: restricted attacks often result in compromised image quality and limited stealthiness, whereas unrestricted attacks risk degrading transferability. To address this challenge, this paper proposes the latent-space adversarial diffusion attack framework (LADA), which focuses on balancing stealthiness and transferability in adversarial attacks against RSI models. LADA employs a pre-trained diffusion model to map high-resolution RSIs into a low-dimensional latent space, enabling semantic-level perturbation optimization while avoiding pixel-wise explicit noise. Additionally, text prompts are automatically generated using a large multimodal model to guide adversarial sample synthesis, ensuring semantic consistency. To enhance perturbation search efficiency in the latent space, a hybrid strategy combining multi-scale sampling and covariance matrix adaptation evolution strategy is introduced. Extensive experiments demonstrate that LADA achieves superior performance across multiple RSI datasets, model architectures, and defense mechanisms. This paper establishes a benchmark for high-stealthiness and high-transferability adversarial attacks, advancing the secure deployment of DNNs in remote sensing applications.

We consider the black-box adversarial attack problem in the field of remote sensing images (RSIs) to reveal the vulnerabilities of various deep neural networks (DNNs), including classification and semantic segmentation models. Existing adversarial attack methods typically focus solely on maximizing attack success rates (ASRs) under given perturbation constraints, neglecting the differences between adversarial samples and clean images. We propose a batchwise amplitude spectrum normalization (BAMPN) method, which is a plug-and-play and transfer-based black-box attack strategy. Using Fourier transform, we convert RSIs from the spatial domain into the frequency domain to obtain the amplitude spectrum, which is normalized within the batch. Moreover, we use a moving average strategy to retain the historical amplitudes of the batch, enhancing input diversity. By mixing the low-level statistical features of RSIs, BAMPN reduces the differences between adversarial samples and clean images while improving attack transferability. In addition, BAMPN is applicable not only to RSIs classification tasks but also directly to semantic segmentation tasks, as it preserves the spatial semantic structure of RSIs. We conduct extensive experiments using 20 DNN models and four benchmark RSIs’ datasets, comparing our method against 14 state-of-the-art (SOTA) approaches. The results demonstrate that BAMPN achieves superior attack performance while ensuring a promising similarity between adversarial and clean samples.

Existing deep learning-based building change detection (BCD) methods are often hindered by sample imbalance and imagery noise, which leads to inaccurate predictions, particularly for building edges and minority changed class regions. To overcome these limitations, we propose a novel Uncertainty-guided Bayesian Adaptation (UBA) framework, designed as a plug-and-play module to enhance existing BCD methods. The UBA framework consists of two core components. First, a Local Bayesian Adaptation strategy (LBs) pragmatically adapts the output layer of any BCD network, enabling efficient prediction uncertainty estimation and decomposition. We demonstrate that the decomposed aleatoric and epistemic uncertainty terms semantically highlight building edges and minority changed class regions, respectively. Based on this insight, we propose an Uncertainty-Weighted Optimization Mechanism (U-Wom) that leverages these uncertainty maps to dynamically re-weight the loss function. This mechanism guides the model to focus its learning on these challenging, fine-grained regions. Extensive experiments on several widely-used BCD datasets show that the UBA framework consistently and significantly improves the performance of various state-of-the-art methods.

With the rapid advancements in computer vision, high-resolution remote sensing imagery has become a crucial data source for object detection. Nevertheless, effectively utilizing limited computational resources and reducing the burden on satellite edge devices remains a significant challenge. To effectively reduce model complexity while maintaining its representational capacity, this article proposes a scale-aware pruning framework (SAPF) to enhance remote sensing object detection ability. First, this article classifies the convolutional layers in object detection models into two categories: layers with a single-scale feature representation and layers with a multiscale feature representation. For convolutional layers with single-scale features, we utilize singular value decomposition (SVD) to quantify feature importance and assess filter redundancy to enhance model efficiency. By removing less critical filters, this pruning criteria aims to reduce the model size and computational load without compromising performance. However, convolutional layers with multiscale features are crucial for optimizing feature extraction and balancing information capture across various scales. To address this, this article evaluates the similarity between convolutional layers with different scales to determine the contribution of various scale features in multiscale fusion. Surprisingly, the SAPF can reduce the FLOPs and parameters, as well as ensure the representational ability obviously when the YOLO v5s and Faster-RCNN are adopted to classify the NWPU VHR-10, RSOD, and SIMD datasets. This means we can save the training computation resources for the model. Additionally, SAPF can significantly improve the efficiency of the model in object detection to ensure its real-time performance.

Using multimodal remote sensing images for change detection (CD) can significantly improve the feasibility and reliability in challenging environments. However, the differences in imaging mechanisms make multimodal images highly heterogeneous. A key challenge for multimodal CD (MCD) is that the heterogeneity of the modalities and changes in ground objects are intertwined during processing. To address this issue, this article proposes a change masked modality alignment network (CMMAN), which uses a multitask framework consisting of one CD branch and two image modal transformation (IMT) branches. Specifically, to ensure a unified feature space, bi-temporal multimodal images are first input into the same Swin-Transformer-based encoder. The extracted features are then fed simultaneously into the CD branch and separately into the two IMT branches. In the CD branch, the decoder is also designed based on the Swin-Transformer, and a weakly modality-correlated feature enhancement (WMCFE) module is introduced to mitigate the interference of modality heterogeneity on CD. For the two IMT branches, both employ a generative adversarial network (GAN) to transform between modalities, and the distributions of features from different modalities are aligned through simultaneous optimization. Uniquely, the change probability map predicted by the CD branch is utilized to mask the change regions in IMT, further decoupling ground object changes and modal heterogeneity. Experimental results on multiple public datasets demonstrate that the proposed CMMAN significantly improves MCD performance and shows good compatibility and portability with various common backbone networks.

Detecting land cover changes from multi-temporal and multi-modal remote sensing images acquired by different sensors at the same location is a complex yet highly valuable task. Recently, diffusion models, exemplified by the Denoising Diffusion Probabilistic Model (DDPM), have garnered significant attention for their remarkable performance and straightforward architecture. These models excel in image generation, distribution modeling, and feature extraction, making them highly promising for advancing Multimodal Change Detection (MCD). In this paper, we propose a Dual-stream Denoising Diffusion Probabilistic Model (D3PM) to address the challenges of MCD. Specifically, D3PM leverages DDPM to design two distinct processing streams, one for each image modality. The first stream employs an unconditional DDPM, whose denoising encoder-decoder network can achieve robust feature extraction. The second stream employs a conditional DDPM to facilitate modal translation, enabling the extracted features to align with the characteristics of the other modality, thereby improving cross-modality comparability. To further enhance performance, we constructed a CD task branch based on the decoder features of the two DDPMs across multiple denoising time steps. Additionally, we designed a collaborative learning optimization strategy with asynchronous time steps, fostering cross-task knowledge sharing and mutual enhancement while preserving the integrity of individual task learning. Experimental results on multiple public datasets demonstrate the effectiveness and superiority of the proposed D3PM, which achieves efficient modal transformation and alignment, mitigates modal heterogeneity interference, and significantly improves detection performance.

Hyperspectral images (HSIs) capture extensive spatial and spectral information, facilitating detailed change detection (CD) of complex land covers. However, the high correlation among spectral data can lead to information redundancy, increasing processing dimensions and introducing irrelevant or detrimental data to CD. To address these challenges, we propose an adaptive center-focused hybrid attention network (ACFHAN) for CD in HSIs. This network adaptively emphasizes the spatial regions and spectral channels most pertinent to CD while suppressing irrelevant information. The architecture establishes an end-to-end mapping from the two HSIs to the change results, featuring multiple center-focused hybrid attention blocks (CFHABs). Each CFHAB integrates two different attention modules, including an adaptive spatial–spectral hybrid self-attention (S2HSA) module that dynamically adjusts spatial–spectral feature weights and a center-focused attention (CFA) module that enhances the area most relevant to the center pixel to be classified. Additionally, to tackle the challenges of expensive labeling, we further designed a multiscale superpixel-based data augmentation method which combines traditional unsupervised and supervised methods to provide sufficient low-cost but high-confidence labeled data for CD. Experimental results across various HSI CD datasets validate the effectiveness of our proposed method.

Remote sensing change detection plays a critical role in urban land management and disaster assessment. However, most existing methods rely on expensive and time-consuming pixel-level labels, limiting their practical applicability. Weakly supervised change detection methods, such as only using image-level labels, offer the potential to reduce annotation costs while maintaining robust detection performance. However, this coarse supervisory information often makes it difficult to accurately capture fine-grained details, resulting in poor pixel-level detection accuracy. To overcome these challenges, we propose a novel Boundary-Aware Refinement Network (BARNet) for weakly supervised change detection, which utilizes a two-stage framework that first generates pixel-level pseudo labels via image-level CD activation maps, then subsequently trains a pixel-level CD network using these generated pseudo labels. Specifically, the first stage adopted a teacher-student distillation image-level CD network, which integrated a multi-scale boundary feature attention module, along with activation ambiguity loss and contrastive learning loss as feature separation constraints, to generate high-quality pseudo labels. In the second stage, these pseudo labels are used to provide deep supervision a pixel-level CD network, where the boundary-aware decoupling module further refines boundary information, leading to more precise segmentation of change areas. Extensive experiments on three public datasets demonstrate that BARNet not only achieves state-of-the-art performance in the weakly supervised change detection domain but also shows competitive performance with existing fully supervised methods, significantly reducing annotation costs while maintaining detection accuracy. With its strong performance, BARNet demonstrates great potential for practical applications in scenarios with limited supervision.

Road change detection (RCD) is crucial for intelligent transportation, disaster assessment, and urban planning. However, current general change detection (CD) methods focus on various targets, such as buildings, while less attention is paid to the CD of narrow and elongated roads. Compared with general CD, RCD may still be limited by the following two aspects: On the one hand, the road usually occupies a small proportion of pixels in remote sensing images (RSIs) and is often easily blocked by buildings, trees, etc., making it difficult to ensure the integrity and connectivity of road structural features in RCD. On the other hand, RCD may easily be confused with the semantic information of similar material backgrounds (such as parking lots and building roofs) due to the lack of salient road features. To overcome the above limitations, we propose a skeleton-guided large-kernel attention Transformer (SLAFormer) for RCD, which can focus on salient road structural and semantic features to enhance its performance. In the proposed SLAFormer, we construct a novel skeleton-guided large-kernel attention module (SLKAM) and a frequency-guided cross spatial-channel difference module (FSCDM) to achieve the above goals. The SLKAM is used to make the model focus on road-specific skeleton features, which preserve the overall structure and morphology of roads to enhance the continuity and integrity of road features. In addition, the FSCDM is devised to better capture small-scale road changes and reduce semantic confusion with similar backgrounds, thereby enhancing change regions and extracting highly discriminative road difference information. Extensive experiments on two public RCD datasets show that ours achieves better RCD accuracy compared with several state-of-the-art (SOTA) approaches. The code will be available at https://github.com/TongfeiLiu/SLAFormer-for-RCD.

For adversarial attacks on aerial image object detection, some physical background attack methods have been proposed and demonstrated excellent performance. However, most of the methods restrict the protected object to a fixed area, and changes in the position and size of the object can affect the effectiveness of the attack. In addition, the size of the background is only related to the size of the object, without considering any reduction in the background area. To alleviate these issues, this paper proposes a novel adversarial background patch generation strategy, which generates an adversarial background patch where aircrafts parked at any position within it remain undetected. Besides we also aim to make the size of the background as small as possible to improve the concealment of the attack and reduce the economic cost. Specifically, a series of adversarial images are generated by placing the aircraft at random angles and positions on the adversarial background patch. Then, a novel background patch optimization strategy is proposed, which enhances the occlusion robustness of the adversarial background patch by weighting the adversarial loss of each image in the batch based on adversarial difficulty. In addition, this paper proposes a novel area loss for achieving optimal area size, which is used to reduce the cost of producing the patch. Finally, a multi-objective optimization method based on Pareto efficiency is introduced for balancing two conflicting losses, the adversarial loss and the area loss. Experimental results show that the adversarial background patches generated by this method have excellent attack effects in both digital and physical attacks, exhibiting strong occlusion robustness. In addition of that, the adversarial background patches generated by this method achieve effective attacks in a smaller area and reduce the physical implementation cost.

Remote sensing (RS) scenarios usually involve sensitive geographic information on national security and regional development. In the commonly used centralized machine-learning paradigm, data dispersed in various locations are concentrated and processed on a single server, which is prone to privacy leakage and data security concerns. Besides, it is difficult to solve the high heterogeneity of RS images by simply applying federated learning (FL) algorithms to scene classification. In this article, we formulate a federated remote sensing scene classification (FedSC) framework, and design a customized neural architecture search (CNAS) to achieve both global generality for multiparty collaborative distributed training and local specificity for personalized RS scene customization. The proposed FedSC is generalizable to be implemented in any manually designed networks, network pruning strategies, or NAS methods related to remote sensing scene classification (RSSC). While the designed CNAS not only achieves collaborative distributed training in protecting participant data privacy to obtain a generalized global model, but also provides a customized local model for each participant that is more in line with the characteristics of private RS scenarios. Overall, the proposed FedSC$_{\textrm {CNAS}}$provides a novel federated collaborative training paradigm for RSSC in terms of data privacy, data heterogeneity, and personalized customization. Extensive analytical and comparative experiments on three benchmark RSSC datasets validate the versatility and effectiveness of our methods, and the proposed FedSC$_{\textrm {CNAS}}$exhibits superior competitiveness compared to state-of-the-art methods.

Multimodal change detection (MCD) has garnered significant interest due to its capacity to address a variety of emergencies in a timely and effective manner. However, discrepancies in sensors and imaging techniques often hinder the direct comparison of heterogeneous remote sensing images (HRSIs), making it difficult to extract change information. To overcome this challenge, we propose a novel superpixel-based AutoEncoder Kolmogorov-Arnold Network (AEKAN) for unsupervised MCD. The primary objective of AEKAN is to excavate the latent commonality features between HRSIs. Notably, commonality features in unchanged regions are generally more pronounced than those in changed regions, which can be leveraged to assess change magnitude. To achieve this, the proposed method utilizes the Kolmogorov-Arnold Network (KAN), renowned for its capability to model data distributions, to extract these commonality features between HRSIs. Concretely, the proposed AEKAN consists of a Siamese KAN encoder and dual KAN decoders. The Siamese encoder aims to map HRSIs and extract latent commonality features, while the dual decoders reconstruct original bitemporal images from these features. In addition, we incorporate a hierarchical commonality loss function within the Siamese encoder to train AEKAN. This loss function is designed to intentionally guide the network in capturing commonality features by minimizing the discrepancies in features extracted from HRSIs at each layer of the Siamese encoder. The extracted commonality features are then adopted to quantify the change magnitude between images through mean square error (MSE). Extensive experiments on five MCD datasets demonstrate that the proposed AEKAN outperforms existing methods. The source code is available at:https://github.com/TongfeiLiu/AEKAN-for-MCD.

The advancement of Convolutional Neural Networks (CNNs) has enhanced remote sensing scene classification on satellites. However, the increased computational complexity of CNNs will impose a substantial burden on satellite hardware, thereby hindering the practical application. Although various pruning techniques have been developed to reduce the scale of CNNs by assessing the importance of model parameters, these weight-based methods often lead to a degradation in model performance when the original model fails to provide meaningful parameters at under-trained conditions. In this paper, we introduce a novel Layer-interaction Adaptive Pruning (LiAP) method designed to streamline under-trained models. Unlike conventional approaches, LiAP evaluates the importance of neurons based on the distribution of eigenvalues rather than individual parameters. Specifically, this is achieved by projecting the weight matrix of each convolutional layer into the eigenspace, where the eigenvalues are utilized to assess the redundancy of filters. Then each space will be assigned a fined score based on the eigenvalue distribution to provide a robust measure of filter importance. Compared to weight-based pruning methods, LiAP maintains consistent evaluation accuracy between well-trained and under-trained models, as the eigenspace is inherently robust to variations in the weight parameter space. We conducted extensive experiments on VGG-16 and ResNet-50 architectures using datasets such as AID, NWPU-RESISC45, PatternNet, and WHU-RS19 over various data partitions. Notably, our method achieved State-of-the-Art (SOTA) reductions in both FLOPs and parameters for VGG-16 across the aforementioned datasets. These results underscore the efficacy of LiAP in enhancing the efficiency and applicability of CNNs in satellite-based remote sensing tasks.

Multimodal change detection (MCD), as an emerging task, aims at recognizing change regions from bi-temporal remote sensing images (RSI) of different modalities. Inspired by the success of the self-attention mechanism in transformer, attempts have been made to solve MCD through the transformer variants. However, transformer-based network optimization requires high-quality training samples. In addition, due to the significant differences in the data distribution, semantic information, and feature representation of multimodal data, transformer-based methods have obvious deficiencies in local feature representation and spatial consistency, especially when dealing with heterogeneous images. To address the above challenges, we propose a collaborative frequency-aware transformer for MCD (CFAT-MCD). As an unsupervised framework, CFAT-MCD is capable of learning more fine-grained patterns of land cover change through a few pseudo-labels. The CFAT is designed to enhance spatial consistency and align the features on a multi-scale basis, which can effectively mitigate the effects of modal differences. In addition, we propose a window-based spatial-frequency collaborative representation (SFCR) module to introduce frequency information into the spatial domain and improve the discriminability of spatial features. Extensive experiments on public datasets and quantitative analyses have validated the superior detection performance of our approach and the effectiveness of each module.

Synthetic aperture radar (SAR) is widely recognized as an effective tool for investigating oceanic processes, particularly internal wave dynamics, due to its ability to capture high-resolution imaging across diverse environmental conditions. However, speckle noise in SAR imagery poses significant challenges for accurately extracting critical parameters, such as wavefront characteristics and bright-dark distances, thereby hindering its application in detailed internal wave studies. To address this, a novel speckle filtering method is proposed, specifically designed to enhance the quality of SAR images capturing the feature of internal wave. First, a structural description matrix with complex values is introduced to encode pixel value correlations and local spatial features, enabling improved representation of wave-related contextual information within the SAR data. Second, a similarity testing framework based on the Wishart distribution is developed to identify and aggregate structurally similar regions, ensuring robust noise suppression while preserving essential structural parameters. The proposed method strikes a balance between speckle noise reduction and structural preservation, making it adaptable to a variety of SAR datasets. Experimental validation on SAR imagery of oceanic internal waves demonstrates the method’s effectiveness in retaining key internal wave information, while additional tests on terrestrial targets highlight its potential for generalization across different scenarios.

Diffusion model has achieved excellent performance in natural image processing, which can learn the noise distribution by the degradation and restoration processes. However, the model is limited to Gaussian noises. Actually, Polarimetric Synthetic Aperture Radar(PolSAR) images have complex non-Gaussian speckle noises, for which the Gaussian diffusion model is difficult to learn their intrinsic statistical characteristics. In this paper, we propose a novel scattering mechanism inspired non-Gaussian diffusion model for PolSAR image classification. To better simulate the PolSAR speckle noise, a mixed noise distribution is defined for PolSAR covariance matrices by combining Gamma multiplicative and Gaussian additive noises. A non-Gaussian forward noising process is derived to degrade a clean PolSAR image to a noisy image by steps. Then, the U-net structure is trained to remove noises for each step, effectively extracting non-Gaussian statistical features. However, statistical features can only characterize the overall distribution of the dataset, which is insufficient to describe complicated individual objects; the original PolSAR data reflect the detailed scattering mechanism for individual pixels, which can provide complementary object information for classification. Therefore, a scattering-statistical joint learning network is further developed with a dual-branch architecture to enhance discrimination ability. In particular, a multiscale pyramid module and attention mechanism are designed to improve the ability of feature learning. Experimental results on five real PolSAR datasets demonstrate that the proposed method effectively captures edge details and preserves homogeneous regions for terrain classification, especially in heterogeneous regions.

In recent years, deep learning (DL) has achieved remarkable success in classifying hyperspectral images (HSIs), relying heavily on the quantity and quality of labeled samples. However, obtaining sufficient labels for HSIs poses a challenge. HSIs obtained by the same sensor often exhibit similar spectral information due to their shared physical, chemical properties, or reflective attributes. Joint analysis of several HSIs enables the integration of limited labeled samples and extraction of more robust and discriminative features from different HSIs. Therefore, a multitask collaborative network (MTCN) for the joint classification of multiple HSIs acquired by the same sensor in different areas is proposed. In the MTCN, each HSI has its own feature extraction channel, which facilitates the learning of image-specific representations. In addition, a feature sharing channel (FSC) is created to extract and transfer multihierarchical image-shared representations between multiple HSIs, thereby forming a common knowledge pool to facilitate feature sharing. Furthermore, a cross-channel mutual attention module (CMAM) is designed to collaboratively utilize features from image-specific and image-shared channels, enhancing the efficiency of information communication in HSIs. The experimental results on six HSIs demonstrate that the proposed MTCN can jointly classify multiple HSIs by the same sensor in different areas and achieve good classification performance.

Memory stability and learning flexibility in continual learning (CL) is a core challenge for cross-scene Hyperspectral Anomaly Detection (HAD) task. Biological neural networks can actively forget history knowledge that conflicts with the learning of new experiences by regulating learning-triggered synaptic expansion and synaptic convergence. Inspired by this phenomenon, we propose a novel Biologically-Inspired Continual Learning Generative Adversarial Network (CL-BioGAN) for augmenting continuous distribution fitting ability for cross-domain HAD task, where Continual Learning Bio-inspired Loss (CL-Bio Loss) and self-attention Generative Adversarial Network (BioGAN) are incorporated to realize forgetting history knowledge as well as involving replay strategy in the proposed BioGAN. Specifically, a novel Bio-Inspired Loss composed with an Active Forgetting Loss (AF Loss) and a CL loss is designed to realize parameters releasing and enhancing between new task and history tasks from a Bayesian perspective. Meanwhile, BioGAN loss with L2-Norm enhances self-attention (SA) to further balance the stability and flexibility for better fitting background distribution for open scenario HAD (OHAD) tasks. Experiment results underscore that the proposed CL-BioGAN can achieve more robust and satisfying accuracy for cross-domain HAD with fewer parameters and computation cost. This dual contribution not only elevates CL performance but also offers new insights into neural adaptation mechanisms in OHAD task.

In the above article [1], in (19), the original formulation has been revised and replaced with the following: \begin{equation*} \textbf {R}_{t}\gets \ \textbf {R}_{t-1}\cup \ k\left ({{\textbf B_{t}}}\right ),\textbf {R}_{0}=\phi . {5pt}\tag {19}\end{equation*}

Hyperspectral imaging has demonstrated substantial advantages in enhancing classification performance in remote sensing applications due to its abundant spectral information. To address the challenges of label noise and class imbalance in hyperspectral image (HSI) classification, we propose an end-to-end Feature-Guided Network (FGN) for HSI. Instead of merely combining spatial and channel attention, FGN leverages feature-level attention interactions to enhance contextual understanding, leading to better feature extraction, especially for underrepresented classes. Furthermore, a bidirectional loss for curriculum learning (CL) is proposed to rank the HSI training data in a descending or ascending order. The top and bottom loss regularizers are designed to make the proposed model suitable for noisy and imbalanced HSI data distributions. In the phase of selecting pace parameter, a stacking ensemble curriculum learning (SECL) model is established to avoid that the outliers and noisy HSI data are involved into the CL training process. A novel instruction matrix based on sample weights is designed for base classifiers. The outputs of the base models, combined with the expected labels, form the input-output pairs for training the second-level classifier. Experiments conducted on multiple hyperspectral imbalanced datasets with noisy labels demonstrate the superior performance of our method.

In the remote sensing community, heterogeneous images captured by various observation platforms usually exhibit different visual appearances and statistical distribution characteristics, making it difficult to detect land surface changes through direct comparison. Most existing change detection (CD) methods aim to extract change information by transforming heterogeneous images into a common image domain or high dimensional feature space for change analysis, neglecting the deep mining of image domain-invariant features. To overcome this challenge, a novel unsupervised cross-domain difference representation learning (CDRL) framework is proposed for heterogeneous CD, including an image translation network and a CD network. First, the image translation network combines within-domain self-reconstruction and cross-domain image translation with CD constraints to enable joint optimization, thereby constructing a style-independent, content-comparable feature space for obtaining difference information between heterogeneous images. Subsequently, patch-based samples with reliable labels are selected by analyzing the difference information through thresholding. On this basis, a simple yet efficient CD network is established by partially reusing the content feature extraction module and incorporating the difference information fusion module. This design enables the network to effectively learn the semantics of both changed and unchanged pixels, thus accurately identifying the ground changes. Extensive experimental results on five different types of datasets demonstrate the effectiveness of the proposed method, showing remarkable improvements over state-of-the-art approaches in terms of accuracy and efficacy. The code and dataset are available at https://github.com/OMEGA-RS/CDRL.

The application of remote sensing (RS) for building change detection (BCD) is indispensable for assessing shifts in land use and surface dynamics. Nevertheless, off-the-shelf deep learning-based BCD methods often suffer from incomplete change boundaries and pseudo changes, due to the insufficient utilization of difference information from bitemporal remote sensing images. To address these issues, we propose a difference-aware multiscale feature aggregation network (DMFANet) aimed at investigating more representative forms of change representation in BCD tasks. To facilitate comprehensive bitemporal feature alignment and differencing, the feature modulation module (FMM) is designed, which focuses on developing semantically robust and contextually enriched pyramidal feature representations by channel-spatial modulation. Subsequently, the cross-domain difference enhancement module (CDEM) is introduced to accurately locate changed building areas with intricate details by capturing multi-perspective difference information through the construction of different domain dependencies. Moreover, we propose a multi-scale context aggregation module (MCAM) to effectively adapt to building scale variations by aggregating multiscale difference features under the guidance of contextual information while mitigating the interference of redundant difference information. The empirical outcomes firmly confirm the superiority of our stream-lined network over nine cutting-edge approaches on the LEVIR-CD, WHU-CD, and SYSU-CD benchmarks, excelling both in terms of accuracy and efficiency. Code and pretrained models are accessible at https://github.com/SallyRonionGit/DMFANet.

Deep learning-based methods for hyperspectral image super-resolution (SR) have achieved significant success in recent years. These methods typically consist of feature extraction module (FEM) and upsampling module. However, due to structural limitations of the upsampling module, most current methods focus on training separate models for different scale factors, which ignores the exploration of potential feature interdependence among different scale factors. In response to these challenges, we introduce a novel framework, called “meta-collaborative learning for arbitrarily scaled hyperspectral image super-resolution” (MCArb). Specifically, MCArb integrates a collaborative learning framework with a meta-learning-based 3-D upsampling module (3DMetaUM) and a scale-aware feature adaptation module (SAFAM). It enables training multiple SR tasks at different scale factors within a single network at the same time. This strategy is able not only to process arbitrary-scale-factor SR for hyperspectral images but also to harness the latent feature interdependence among different scales. In this study, we applied the MCArb framework to transform three deep learning-based hyperspectral image SR networks to MCArb methods, resulting in significant performance enhancements across five hyperspectral datasets. These improvements showcase the proposed MCArb framework’s ability to enhance feature extraction efficiency and capitalize on latent interscale correlations. This code is available athttps://github.com/ShuangWu-XDU/MCArb_HSI_SR.

Federated learning (FL) is a distributed learning paradigm that leverages local updates and parameter sharing to address privacy concerns in Industrial Internet of Things (IIoT) environments. The presence of statistical heterogeneity among IIoT devices poses significant challenges for FL, impacting convergence and model performance. Although previous studies have attempted to mitigate this issue, a fundamental solution remains elusive. To address this, we propose a novel approach called federated multidiscriminators multigenerators generative adversarial network (FedMDMG-GAN), which employs a distributed generative adversarial network (GAN). IIoT devices concurrently train local generators and discriminators to generate data with global information. The server then aggregates parameters and redistributes them to the devices to refine local GANs. In addition, we introduce a proximal term for global aggregation to enhance convergence. Theoretical analysis suggests that the FedMDMG-GAN algorithm can asymptotically converge to a stable point. Qualitative assessments demonstrate that our method can generate images closely resembling real data with comprehensive global information. Quantitative results indicate that FedMDMG-GAN outperforms vanilla FL and state-of-the-art methods.

The main challenge of multimodal change detection (MCD) is that multimodal bitemporal images (MBIs) cannot be compared directly to identify changes. To overcome this problem, this paper proposes a novel commonality feature representation learning (CFRL) and constructs a CFRL-based unsupervised MCD framework. The CFRL is composed of a Siamese-based encoder and two decoders. First, the Siamese-based encoder can map original MBIs in the same feature space for extracting the representative features of each modality. Then, the two decoders are used to reconstruct the original MBIs by regressing themselves, respectively. Meanwhile, we swap the decoders to reconstruct the pseudo-MBIs to conduct modality alignment. Subsequently, all reconstructed images are input to the Siamese-based encoder again to map them in a same feature space, by which representative features are obtained. On this basis, latent commonality features between MBIs can be extracted by minimizing the distance between these representative features. These latent commonality features are comparable and can be used to identify changes. Notably, the proposed CFRL can be performed simultaneously in two modalities corresponding to MBIs. Therefore, two change magnitude images (CMIs) can be generated simultaneously by measuring the difference between the commonality features of MBIs. Finally, a simple threshold algorithm or a clustering algorithm can be employed to divide CMIs into binary change maps. Extensive experiments on six publicly available MCD datasets show that the proposed CFRL-based framework can achieve superior performance compared with other state-of-the-art approaches.

Existing free-energy guided No-Reference Image Quality Assessment (NR-IQA) methods continue to face challenges in effectively restoring complexly distorted images. The features guiding the main network for quality assessment lack interpretability, and efficiently leveraging high-level feature information remains a significant challenge. As a novel class of state-of-the-art (SOTA) generative model, the diffusion model exhibits the capability to model intricate relationships, enhancing image restoration effectiveness. Moreover, the intermediate variables in the denoising iteration process exhibit clearer and more interpretable meanings for high-level visual information guidance. In view of these, we pioneer the exploration of the diffusion model into the domain of NR-IQA. We design a novel diffusion model for enhancing images with various types of distortions, resulting in higher quality and more interpretable high-level visual information. Our experiments demonstrate that the diffusion model establishes a clear mapping relationship between image reconstruction and image quality scores, which the network learns to guide quality assessment. Finally, to fully leverage high-level visual information, we design two complementary visual branches to collaboratively perform quality evaluation. Extensive experiments are conducted on seven public NR-IQA datasets, and the results demonstrate that the proposed model outperforms SOTA methods for NR-IQA. The codes will be available at https://github.com/handsomewzy/DiffV2IQA.

The empirical studies of most existing graph neural networks (GNNs) broadly take the original node feature and adjacency relationship as single-channel input, ignoring the rich information of multiple graph channels. To circumvent this issue, the multichannel graph analysis framework has been developed to fuse graph information across channels. How to model and integrate shared (i.e., consistency) and channel-specific (i.e., complementarity) information is a key issue in multichannel graph analysis. In this article, we propose a cross-channel graph information bottleneck (CCGIB) principle to maximize the agreement for common representations and the disagreement for channel-specific representations. Under this principle, we formulate the consistency and complementarity information bottleneck (IB) objectives. To enable optimization, a viable approach involves deriving variational lower bound and variational upper bound (VarUB) of mutual information terms, subsequently focusing on optimizing these variational bounds to find the approximate solutions. However, obtaining the lower bounds of cross-channel mutual information objectives proves challenging through direct utilization of variational approximation, primarily due to the independence of the distributions. To address this challenge, we leverage the inherent property of joint distributions and subsequently derive variational bounds to effectively optimize these information objectives. Extensive experiments on graph benchmark datasets demonstrate the superior effectiveness of the proposed method.

Nowadays, pruning techniques have drawn attention to convolutional neural networks (CNNs) for reducing the consumption of computation resources. In particular, the Taylor-based method simplifies the evaluation of importance for each filter as the product of the gradient and weight value of the output features, which outperforms other methods in reductions of parameters and floating point operations (FLOPs). However, the Taylor-based method sacrifices too much accuracy when the overall pruning rate is relatively large compared with other pruning algorithms. In this article, we propose a self-adaptive attention factor (SAAF) to improve the performance of the slimmed model when conventional Taylor-based pruning is utilized under higher pruning. Specifically, SAAF can be calculated by leveraging the remaining ratio of filters at the early pruning stage of the Taylor-based method, and then, some pruned filters can be recovered for improving the accuracy of the slimmed model in terms of SAAF. It means that SAAF can protect filters from being overslimmed to eliminate the degeneration of Taylor-based pruning when the pruning rate is large as well as can compress models apparently across various datasets. We test the efficiency of SAAF on VGG-16 and ResNet-50 with CIFAR-10, Tiny-ImageNet, ImageNet-1000, and remote sensing images. Our method outperforms the traditional Taylor-based method obviously in accuracy, and there are only tiny sacrifices in the reduction of parameters and FLOPs, which is better than other pruning methods.

The precision of unsupervised point cloud registration methods is typically limited by the lack of reliable inlier estimation and self-supervised signal, especially in partially overlapping scenarios. In this article, we propose an effective inlier estimation method for unsupervised point cloud registration by capturing geometric structure consistency between the source point cloud and its corresponding reference point cloud copy. Specifically, to obtain a high-quality reference point cloud copy, a one-nearest neighborhood (1-NN) point cloud is generated by input point cloud, which facilitates matching map construction and allows for integrating dual neighborhood matching scores of 1-NN point cloud and input point cloud to improve matching confidence. Benefiting from the high-quality reference copy, we argue that the neighborhood graph formed by inlier and its neighborhood should have consistency between source point cloud and its corresponding reference copy. Based on this observation, we construct transformation-invariant geometric structure representations and capture geometric structure consistency to score the inlier confidence for estimated correspondences between source point cloud and its reference copy. This strategy can simultaneously provide the reliable self-supervised signals for model optimization. Finally, we further calculate transformation estimation by the weighted SVD algorithm with the estimated correspondences and the corresponding inlier confidence. We train the proposed model in an unsupervised manner, and extensive experiments on synthetic and real-world datasets illustrate the effectiveness of the proposed method.

The few-shot image classification task is to enable a model to identify novel classes by using only a few labeled samples as references. In general, the more knowledge a model has, the more robust it is when facing novel situations. Although directly introducing large amounts of new training data to acquire more knowledge is an attractive solution, it violates the purpose of few-shot learning with respect to reducing dependence on big data. Another viable option is to enable the model to accumulate knowledge more effectively from existing data, i.e., improve the utilization of existing data. In this article, we propose a new data augmentation method called self-mixup (SM) to assemble different augmented instances of the same image, which facilitates the model to more effectively accumulate knowledge from limited training data. In addition to the utilization of data, few-shot learning faces another challenge related to feature extraction. Specifically, existing metric-based few-shot classification methods rely on comparing the extracted features of the novel classes, but the widely adopted downsampling structures in various networks can lead to feature degradation due to the violation of the sampling theorem, and the degraded features are not conducive to robust classification. To alleviate this problem, we propose a calibration-adaptive downsampling (CADS) that calibrates and utilizes the characteristics of different features, which can facilitate robust feature extraction and benefit classification. By improving data utilization and feature extraction, our method shows superior performance on four widely adopted few-shot classification datasets.

The vulnerability to poor local optimum and the memorization of noise data limit the generalizability and reliability of massively parameterized convolutional neural networks (CNNs) on complex real-world data. Self-paced curriculum learning (SPCL), which models the easy-to-hard learning progression from human beings, is considered as a potential savior. In spite of the fact that numerous SPCL solutions have been explored, it still confronts two main challenges exactly in solving deep networks. By virtue of various designed regularizers, existing weighting schemes independent of the learning objective heavily rely on the prior knowledge. In addition, alternative optimization strategy (AOS) enables the tedious iterative training procedure, thus there is still not an efficient framework that integrates the SPCL paradigm well with networks. This article delivers a novel insight that attention mechanism allows for adaptive enhancement in the contribution of diverse instance information to the gradient propagation. Accordingly, we propose a general-purpose deep SPCL paradigm that incorporates the preferences of implicit regularizer for different samples into the network structure with inductive bias, which in turn is formalized as the self-paced curriculum network (SPCNet). Our proposal allows simultaneous online difficulty estimation, adaptive sample selection, and model updating in an end-to-end manner, which significantly facilitates the collaboration of SPCL to deep networks. Experiments on image classification and scene classification tasks demonstrate that our approach surpasses the state-of-the-art schemes and obtains superior performance.

DNN have demonstrated vulnerability to adversarial attacks in object detection tasks. While significant progress has been made in single-spectrum attacks, cross-spectral adversarial attacks remain challenging due to the complex tradeoffs between visible and infrared domains. To address this, an evolutionary multiobjective cross-spectral attack (MoXAttack) framework, for developing adversarial patches in closed-box cross-spectral scenarios is proposed. MoXAttack incorporates a multipopulation constraint-handling technique, which uses both penalty functions and feasibility rules to guide the search process. Spectrum-aware genetic operators are introduced to enhance solution diversity and feasibility. The framework automatically optimizes the smooth to cross-spectral shared patch shape using curvature energy. In addition, MoXAttack utilizes SVD for visible spectrum texture perturbations and adjustable thermal shielding material thickness for infrared spectrum control. Experiments on the LLVIP dataset demonstrate that MoXAttack achieves competitive performance across multiple object detection models. Ablation studies reveal the positive impact of improved components on attack effectiveness. The multipatch strategy improves attack success rates by at least 17%, while optimized patch shapes outperform conventional geometric shapes by at least 25% in terms of mAP drop. In the physical world test, the proposed method shows stability in different viewing angles.

This article proposes a novel evolutionary multitasking optimization (EMTO) paradigm called divide-and-conquer EMTO, which divides the original complex optimization problem into multiple simple optimization tasks and then these tasks are optimized by EMTO concurrently to formulate the resulting solution of the original problem. The main characteristics of divide-and-conquer EMTO are that the considered problem can be divided into multiple small-scale optimization tasks and the optimal solution is the combination of solutions of all tasks. In order to achieve the optimal combined fitness of all tasks, a relative improvement function and an adaptive exploration optimization strategy are designed for dynamic resource allocation across tasks. Finally, a case study on hyperspectral unmixing is investigated in the proposed divide-and-conquer EMTO framework by dividing the hyperspectral image into several homogeneous regions to formulate multiple sparse unmixing tasks. Experiments on benchmark and sparse unmixing problems demonstrate the superiority of divide-and-conquer EMTO.

Surrogate-assisted evolutionary algorithms for addressing expensive optimization problems with both continuous and categorical variables (EOPCCVs) are still in the early stages of development. This study makes significant advancements by leveraging the mixed-variable nature of EOPCCVs in two crucial ways. First, it introduces a novel hybrid approach combining differential evolution and upper confidence bound sampling (DEUCB), designed to explore the mixed search space effectively. Second, a specialized value distance metric (VDM) is proposed, integrating continuous and categorical variables, to enhance the accuracy of the radial basis function (RBF) model approximation. Finally, we present a customized evolutionary expensive optimization algorithm (CEEO), which seamlessly incorporates DEUCB and RBF-VDM into the widely utilized global and local surrogate-assisted evolutionary optimization framework. Experimental results, compared against state-of-the-art counterparts on three distinct sets of benchmark problems and a convolutional neural network hyperparameter optimization task, consistently affirm the efficacy of the proposed CEEO in addressing EOPCCVs. The source code for the proposed CEEO algorithm is available athttps://github.com/CIAM-Group/EvolutionaryAlgorithm_Codes/tree/main/CEEO_Code.

Graph Structure Learning (GSL) has demonstrated considerable potential in the analysis of graph-unknown non-Euclidean data across a wide range of domains. However, constructing an end-to-end graph structure learning model poses a challenge due to the impediment of gradient flow caused by the nearest neighbor sampling strategy. In this paper, we construct a differential graph structure learning model by replacing the non-differentiable nearest neighbor sampling with a differentiable sampling using the reparameterization trick. Under this framework, we argue that the act of sampling nearest neighbors may not invariably be essential, particularly in instances where node features exhibit a significant degree of similarity. To alleviate this issue, the bell-shaped Gaussian Similarity (GauSim) modeling is proposed to sample non-nearest neighbors. To adaptively model the similarity, we further propose Neural Gaussian Similarity (NeuralGauSim) with learnable parameters featuring flexible sampling behaviors. In addition, we develop a scalable method by transferring the large-scale graph to the transition graph to significantly reduce the complexity. Experimental results demonstrate the effectiveness of the proposed methods.

3D Single Object Tracking (SOT) stands a forefront task of computer vision, proving essential for applications like autonomous driving. Sparse and occluded data in scene point clouds introduce variations in the appearance of tracked objects, adding complexity to the task. In this research, we unveil M3SOT, a novel 3D SOT framework, which synergizes multiple input frames (template sets), multiple receptive fields (continuous contexts), and multiple solution spaces (distinct tasks) in ONE model. Remarkably, M3SOT pioneers in modeling temporality, contexts, and tasks directly from point clouds, revisiting a perspective on the key factors influencing SOT. To this end, we design a transformer-based network centered on point cloud targets in the search area, aggregating diverse contextual representations and propagating target cues by employing historical frames. As M3SOT spans varied processing perspectives, we've streamlined the network—trimming its depth and optimizing its structure—to ensure a lightweight and efficient deployment for SOT applications. We posit that, backed by practical construction, M3SOT sidesteps the need for complex frameworks and auxiliary components to deliver sterling results. Extensive experiments on benchmarks such as KITTI, nuScenes, and Waymo Open Dataset demonstrate that M3SOT achieves state-of-the-art performance at 38 FPS. Our code and models are available at https://github.com/ywu0912/TeamCode.git.

Structured pruning techniques have achieved great compression performance on convolutional neural networks for image classification tasks. However, the majority of existing methods are sensitive with respect to the model parameters, and their pruning results may be unsatisfactory when the original model is trained poorly. That is, they need the original model to be fully trained, to obtain useful weight information. This is time-consuming, and makes the effectiveness of the pruning results dependent on the degree of model optimization. To address the above issue, we propose a novel metric named Average Filter Information Entropy (AFIE). It decomposes the weight matrix of each layer into a low-rank space, and quantifies the filter importance based on the distribution of the normalized eigenvalues. Intuitively, the eigenvalues capture the covariance among filters, and therefore could be a good guide for pruning. Since the distribution of eigenvalues is robust to the updating of parameters, AFIE can yield a stable evaluation for the importance of each filter no matter whether the original model is trained fully. We implement our AFIE-based pruning method for three popular CNN models of AlexNet, VGG-16, and ResNet-50, and test them on three widely-used image datasets MNIST, CIFAR-10, and ImageNet, respectively. The experimental results are encouraging. We surprisingly observe that for our methods, even when the original model is trained with only one epoch, the AFIE score of each filter keeps identical to the results when the model is fully-trained. This fully indicates the effectiveness of the proposed pruning method.

Existing hyperspectral image (HSI) super-resolution (SR) methods struggle to effectively capture the complex spectral-spatial relationships and low-level details, while diffusion models represent a promising generative model known for their exceptional performance in modeling complex relations and learning high and low-level visual features. The direct application of diffusion models to HSI SR is hampered by challenges such as difficulties in model convergence and protracted inference time. In this work, we introduce a novel Group-Autoencoder (GAE) framework that synergistically combines with the diffusion model to construct a highly effective HSI SR model (DMGASR). Our proposed GAE framework encodes high-dimensional HSI data into low-dimensional latent space where the diffusion model works, thereby alleviating the difficulty of training the diffusion model while maintaining band correlation and considerably reducing inference time. Experimental results on both natural and remote sensing hyperspectral datasets demonstrate that the proposed method is superior to other state-of-the-art methods both visually and metrically.

3D point cloud registration is a fundamental task in computer vision, aimed at estimating a transformation to align a pair of point clouds. For point cloud registration, where the popular methods are used to build a compatibility graph. Different methods of constructing compatibility graphs can result in different information regions to be searched in the graph, due to varying constraints. Evolutionary multitask optimization has gained attention in the field of evolutionary computation, as it enables knowledge transfer among multitasks to enhance the exploration of information. Inspired by this theory, this paper proposes a method to utilize compatibility graphs as tasks through evolutionary multitasking for solving the problem of point cloud registration. We first construct two compatibility graph tasks with different tightness constraints for point cloud registration. Due to the different constraints of the two proposed tasks, the local information of the graph of interest might be biased. This bias can be effectively utilized in the evolutionary multitasking framework to enhance the ability to discover meaningful consensus relationships in the search space of the graph. Then we map the two tasks to a unified search space by designing clique selection strategies, and carrying out knowledge transfer between the two tasks, aiming to emphasize more on the local consensus information in the graphs. Lastly, the efficacy of our proposed method is further validated on multiple registration datasets.

Novel benchmarks for multiobjective neural archi-tecture search are emerging consistently. It stimulates the need of knowledge transfer techniques to facilitate the repetitive and cumbersome search process. One-shot surrogate is hereby proposed to transfer knowledge from source problems. Specif-ically, a Pareto-aware super-surrogate construction technique is proposed aiming at efficient exploitation of knowledge from benchmarks to make transfer easier. Then, during the surrogate prediction process, appropriate sub-surrogates are sampled from the super-surrogate to jointly guide the evaluation decision. The multiobjective surrogate is also redesigned as objective-wise, decision-making, and combinative ones, so that it can make transfer flexible, as well as evade from the scalarization issue in performance measurement. This knowledge transfer scheme assists the convergence of the search process using previously constructed surrogates from multiple source problems, balancing exploration and exploitation.

With the acquisition of large-scale remote sensing data and the development of deep learning, convolutional neural networks have achieved great progress in scene clas-sification tasks. However, the current networks greatly rely on the experience design of experts, and a single network cannot cope with multiple complex scene categories. In this paper, we design a novel evolutionary multitasking collaborative neural architecture search (EMCNAS) for remote sensing scene classification. EMCNAS mainly explores the similar features of different remote sensing scene classification tasks, and utilizes the uniformly encoded population to achieve implicit collaborative transfer. EM CNAS is able to adaptively determine the degree of exchange of genetic material based on the similarity of tasks in different scenarios, enabling more effective positive transfer. Compared with excellent manually designed neural networks and NAS peers, the proposed EMCNAS achieved competitive results on two remote sensing scene classification benchmark datasets UC Merced LandUse and NWPU-RESISC45 datasets. Ablation experiments also demonstrate the effectiveness of the adaptive collaborative transfer designed in EMCNAS.

Inliers estimation constitutes a pivotal step in partially overlapping point cloud registration. Existing methods broadly obey coordinate-based scheme, where inlier con-fidence is scored through simply capturing coordinate differences in the context. However, this scheme results in massive inlier misinterpretation readily, consequently affecting the registration performance. In this paper, we explore to extend a new definition called inlier confidence calibration (ICC) to alleviate the above issues. Firstly, we provide finely initial correspondences for ICC in order to generate high quality reference point cloud copy corresponding to the source point cloud. In particular, we develop a soft assignment matrix optimization theorem that offers faster speed and greater precision compared to Sinkhorn. Benefiting from the high quality reference copy, we argue the neighborhood patch formed by inlier and its neighborhood should have consistency between source point cloud and its reference copy. Based on this insight, we construct transformation-invariant geometric constraints and capture geometric structure consistency to calibrate inlier confidence for estimated correspondences between source point cloud and its reference copy. Finally, transformation is further calculated by the weighted SVD algorithm with the calibrated inlier confidence. Our model is trained in an unsupervised manner, and extensive experiments on synthetic and real-world datasets illustrate the effectiveness of the proposed method.

Point cloud registration is a hot research topic in the field of computer vision. In recent years, the registration method based on evolutionary computation has attracted more and more attention because of its robustness to initial pose and flexibility of objective function design. However, most of the current evolutionary computation-based point cloud registration methods do not take into account the multi-view problem, that is, to capture the close relationship between point clouds from different perspectives. We fully realize that if these relations are used correctly, the registration performance can be improved. Therefore, this paper proposes an evolutionary multitasking multi-view point cloud registration method, which solves the problem of multi-view error accumulation. To ensure the unity of global and local, a two-level knowledge transfer strategy is proposed, which divides the multi-view cloud registration task into two levels. This strategy unifies the search space of two registration tasks, solves the negative transfer phenomenon, and avoids the problem of falling into the local optimum. Finally, the effectiveness of the method is verified by sufficient experiments. This method has strong robustness to noise and outliers, and can be effectively implemented in various registration scenarios.

Multi-instance point cloud registration is the problem of estimating multiple rigid transformations between two point clouds. Existing solutions rely on global spatial consistency of ambiguity and the time-consuming clustering of highdimensional correspondence features, making it difficult to handle registration scenarios where multiple instances overlap. To address these problems, we propose a maximal clique based multiinstance point cloud registration framework called PointMC. The key idea is to search for maximal cliques on the correspondence compatibility graph to estimate multiple transformations, and cluster these transformations into clusters corresponding to different instances to efficiently and accurately estimate all poses. PointMC leverages a correspondence embedding module that relies on local spatial consistency to effectively eliminate outliers, and the extracted discriminative features empower the network to circumvent missed pose detection in scenarios involving multiple overlapping instances. We conduct comprehensive experiments on both synthetic and real-world datasets, and the results show that the proposed PointMC yields remarkable performance improvements.

Recent studies have highlighted the effectiveness of deep domain adaptation (DA) techniques in addressing cross-scene hyperspectral image (HSI) classification challenges. However, most of the existing DA methods often prioritize aligning data distributions while overlooking the intrinsic separability between source and target domain data. In this paper, we propose a decoupled contrastive learning based unsupervised domain adaptation (DCLUDA) method for HSI classification. Unlike conventional adversarial DA methods, our method introduces a unique DA loss specifically designed to minimize class confusion in the target domain. This not only simplifies model training but also enhances class discriminability. Moreover, we employ a decoupled contrastive learning strategy on both domains to enhance data separability within each domain. Finally, we propose a sample selection strategy based on confident learning to select high-confidence samples from the target domain for fine-tuning the DA model. Experiments on two cross-scene HSI classification tasks shown that our proposed DCLUDA outperforms several existing DA methods.

The exponential growth of the Internet of Things (IoT), driven by the increasing number of connected devices and sensors, is profoundly transforming many industries. This proliferation has been enabled by advances in storage and processing capabilities, facilitating the implementation of deep model. However, concerns surrounding privacy and substantial bandwidth required for data transmission arise from the centralized processing of data collected from numerous distributed devices. To address these issues, federated learning (FL) has emerged as a viable solution with a decentralized strategy. It ensures data privacy by allowing devices to locally train models while keeping their data localized, thus reducing centralization needs and concerns related to privacy and bandwidth. To mitigate the data heterogeneity problem in FL scenarios, we propose federated pattern extraction clustering (FedPEC) to cluster clients with similar data distributions. Compare to existing client clustering methods, FedPEC requires no additional data transmission and ensures great flexibility, scalability and privacy. Furthermore, we discuss the discrepancy between representation and data distribution in existing methods from the perspective of pattern representation. Based on this, we propose a variant of FedPEC as a solution, which has achieved excellent performance on multiple FL data sets.

Federated learning (FL) encounters two major obstacles, namely, the heterogeneity among Internet of Things (IoT) devices hampers both personalized and generalized performance, and the limited communication resources impede the learning efficiency. This article proposes a decentralized FL approach based on prototype representation learning (called DeProFL) and time-varying communication topology. The heterogeneity among devices can arise from varying data distributions and model architectures, leading to local or global gradient drift. To mitigate the effects of heterogeneity, we introduce prototype learning to establish consistent representations of samples and models across devices. Rather than directly propagating the large model weights, which require significant transmission volumes, we propagate prototype representations among local devices to reduce transmission. Given the limited communication resources of IoT and the dynamic wireless environments, we propose a time-varying decentralized FL approach to address these practical constraints. During each global iteration, every device shares its prototype only with adjacent devices for the collaborative learning, aiming to improve system stability and reduce network bandwidth usage. We have theoretically analyzed and verified the convergence of the DeProFL under nonconvex conditions. Extensive experiments have been applied to the benchmark data sets, and the results show that DeProFL outperforms state-of-the-art methods.

We consider a federated semi-supervised learning (FSSL) scenario in heterogeneous Internet of Things (IoT) healthcare systems where only a tiny fraction of IoT healthcare devices possess labels. The challenge is that most IoT devices need label information and exhibit heterogeneous data distributions. This article proposes a unified FSSL framework (FSSL-HD) tailored for IoT healthcare systems with heterogeneous distributions. The FSSL-HD framework introduces a novel objective function for unlabeled healthcare devices to enhance generalization performance and reduce intermodel discrepancies. A straightforward loss-based aggregation mechanism is designed to reinforce distillation from labeled healthcare devices to supervise unlabeled samples. Multiple iterations for labeled devices and dynamically adjusted confidence thresholds are proposed to improve the model performance. Our theoretical analysis of the generalization error in FSSL suggests directions for enhancing performance by improving the self-training capabilities of unlabeled devices and reinforcing the distillation and transfer of supervision signals from labeled devices. Our empirical experiments on the federated benchmark data sets and the medical image data sets show that FSSL-HD surpasses the performance of state-of-the-art methods. The code is available athttps://github.com/baoshengli96/FSSL-HD.

Convolutional neural networks (CNNs) are used comprehensively in the field of the Internet of Things (IoTs), such as mobile phones, surveillance, and satellite. However, the deployment of CNNs is difficult because the structure of hand-designed networks is complicated. Therefore, we propose a sensitiveness-based network pruning framework (SNPF) to reduce the size of original networks to save computation resources. SNPF will evaluate the importance of each convolutional layer by the reconstruction of inference accuracy when we add extra noise to the original model, and then remove filters in terms of the degree of sensitiveness for each layer. Compared with previous weight-norm-based pruning methods, such as “$\mathscr {C}_{1}$-norm,” “BatchNorm-Pruning,” and “Taylor-Pruning,” SNPF is robust to the update of parameters, which can avoid the inconsistency of evaluation for filters if the parameters of the pretrained model are not fully optimized. Namely, SNPF, can prune the network at the early training stage to save computation resources. We test our method on three prevalent models of VGG-16, ResNet-18, ResNet-50 and a customized Conv-4 with 4 convolutional layers. They are then tested on CIFAR-10, CIFAR-100, ImageNet, and MNIST, respectively. Impressively, we observe that even when the VGG-16 is only trained with 50 epochs, we can get the same evaluation of layer importance as the results when the model is fully trained. Additionally, we can also achieve comparable pruning results to previous weight-oriented methods on the other three models.

State-of-the-art (SOTA) adversarial attacks expose vulnerabilities in object detectors, often resulting in erroneous predictions. However, existing adversarial attacks neglect the stealth and flexibility of adversarial examples, which are crucial for conducting contextually consistent and inconspicuous attacks. To address these issues, leveraging the observed phenomenon of predicted box offsets in real-world object detection scenarios, this paper presents a novel adversarial attack framework called ShiftAttack. It leverages the concept of dense detection in prevalent object detectors, by boosting the confidence of low Intersection over Union (IoU) predictions within the positive samples (the set of predicted boxes responsible for localizing the same target), which leads to the erroneous exclusion of true positive predictions during the post-processing stage. Such a paradigm is highly stealthy as the shifted predictions seem like natural detector mistakes rather than obvious manipulations. To enhance the flexibility of ShiftAttack this paper proposes a generative approach called ShiftAttack Generator (SAG), which can not only shift predicted boxes for any target in arbitrary directions and distances but also facilitate adaptive feature exchange between pre- and post-shift regions to optimize the attack. Additionally, the proposed SAG incorporates the Dynamic Hinge Loss (DHL) to ensure the imperceptibility of perturbations, effectively mitigating the Patch-Pattern associated with the use of$\mathcal {L}_{2}$norm. Extensive experiments confirm that SAG surpasses other SOTA adversarial attacks in effectiveness, speed and stealthiness.

Binary Neural Networks (BNNs) using 1-bit weights and activations are emerging as a promising approach for mobile devices and edge computing platforms. Concurrently, traditional Neural Architecture Search (NAS) has gained widespread usage in automatically designing network architectures. However, the computation involved in binary NAS is more complex than in NAS due to the substantial information loss incurred by binary modules, and different binary spaces are required for different tasks. To address these challenges, a universal binary neural architecture search (UBNAS) algorithm is proposed. In this paper, the ApproxSign function is used to reduce the gradient error and accelerate the convergence in binary network searching and training. Moreover, UBNAS adopts a novel search space consisting of operations appropriate for the binary methods. To improve the original space operation module, we explore the effect of diverse structures for various modules and ultimately obtain a universal binary network structure. Additionally, the channel sampling ratio is adjusted to balance the advantages of different operations and an early stopping strategy is implemented to significantly reduce the computational burden associated with searching. We perform extensive experiments on CIFAR10, and ImageNet datasets and the results demonstrate the effectiveness of the proposed method.

Extracting discriminative representations is the key step for correspondence-free point cloud registration. The extracted representations require to be discriminative to transformation, which demands representations to reduce the influence of redundant information irrelevant to transformation. However, recently proposed methods ignore this crucial property, resulting in limited ability to represent point cloud. In addition, researching correspondence-free point cloud registration has stagnated in recent years. In this paper, we try to relieve features redundancy issue for correspondence-free point cloud registration from a new perspective. Specifically, our method comprises two stages: feature extraction stage and rigid body transformation stage. In feature extraction stage, we aim to maximize multi-hierarchical mutual information between different hierarchical features, which can provide discriminative and less redundancy representations to regress transformation parameters for next stage. In rigid body transformation stage, we utilize dual quaternion to estimate transformation parameters, which combines rotation and translation simultaneously within a unified framework and obtains a compact representations for rigid transformation. The proposed model is trained in an unsupervised manner on the ModelNet40 dataset. The experimental results illustrate that our method achieves higher accuracy and robustness compared with existing correspondence-free methods.

Multiparty learning provides solutions for training joint models with decentralized data under legal and practical constraints. However, traditional multiparty learning approaches are confronted with obstacles, such as system heterogeneity, statistical heterogeneity, and incentive design. Determining how to deal with these challenges and further improve the efficiency and performance of multiparty learning has become an urgent problem to be solved. In this article, we propose a novel contrastive multiparty learning framework for knowledge refinement and sharing with an accountable incentive mechanism. Since the existing parameter averaging method is contradictory to the learning paradigm of neural networks, we simulate the process of human cognition and communication and analogize multiparty learning as a many-to-one knowledge-sharing problem. The approach is capable of integrating the acquired explicit knowledge of each client in a transparent manner without privacy disclosure, and it reduces the dependence on data distribution and communication environments. The proposed scheme achieves significant improvement in model performance in a variety of scenarios, as we demonstrated through experiments on several real-world datasets.

Multiobjective multitasking optimization (MTO) needs to solve a set of multiobjective optimization problems simultaneously, and tries to speed up their solution by transferring useful search experiences across tasks. However, the quality of transfer solutions will significantly impact the transfer effect, which may even deteriorate the optimization performance with an improper selection of transfer solutions. To alleviate this issue, this article suggests a new multiobjective multitasking evolutionary algorithm (MMTEA) with decomposition-based transfer selection, called MMTEA-DTS. In this algorithm, all tasks are first decomposed into a set of subproblems, and then the transfer potential of each solution can be quantified based on the performance improvement ratio of its associated subproblem. Only high-potential solutions are selected to promote knowledge transfer. Moreover, to diversify the transfer of search experiences, a hybrid transfer evolution method is designed in this article. In this way, more diverse search experiences are transferred from high-potential solutions across different tasks to speed up their convergence. Three well-known benchmark suites suggested in the competition of evolutionary MTO and one real-world problem suite are used to verify the effectiveness of MMTEA-DTS. The experiments validate its advantages in solving most of the test problems when compared to five recently proposed MMTEAs.

Evolutionary multitasking optimization (EMTO) is a new optimization paradigm proposed in the field of evolutionary computation in recent years. EMTO can solve several different optimization tasks simultaneously and facilitate superior convergence characteristics by transferring effective knowledge among the tasks. However, existing EMTO usually focuses only on facilitating convergence characteristics while neglecting the potential privacy leakage problem in the knowledge transfer between different tasks. The privacy leakage could result in considerable financial losses or severe reputation impairment, which may impede the development of EMTO in real-world applications. To solve the problem of privacy leakage in EMTO, this paper proposes a privacy-enhanced multitasking particle swarm optimization algorithm. A knowledge transfer strategy with privacy preservation is designed based on homomorphic encryption by combining multitasking particle swarm optimization. In addition, an inter-task knowledge transfer mechanism implemented in a low-dimensional subspace is introduced to reduce the extra computational burden caused by privacy preservation. Comprehensive experiments are conducted on synthetic and NAS problems to verify the effectiveness of the proposed method. According to the experimental results, the proposed method has remarkable advantages in privacy preservation compared to existing EMTO.

Block sparse signals have the characteristics of nonzero values concentrated as blocks. Therefore, block sparse recovery requires recovering the original signal from the observed signal and the measurement matrix, where the recovered error should be small and the signal satisfies the block sparsity. In this article, block sparse recovery is solved as a multiobjective problem (MOP) and the recovery error, sparsity, and the block number of the recovered signal are considered as the conflicting objectives. Furthermore, the dimensionality of real block sparse signals is often too large, which increases the difficulty of recovery. Cooperative coevolutionary (CC) can effectively alleviate the curse of dimensionality in block sparse recovery. In addition, block sparsity causes zero and nonzero variables in the signal to form natural clusters, which is similar to the decomposition and cooperation of subproblems in CC. Therefore, CC is used in the algorithm to solve the block sparse MOP and decomposes the original problem into different subproblems. Each subproblem has a defined nonzero degree that can be combined with the proposed adaptive operator to adaptively adjust the way of generating new solutions of subproblem, which encourages the complete solution to have block sparsity. For the increased number of function evaluations (FEs) caused by the decomposition, a new list-inquiry evaluation is designed to avoid repeated evaluation. Finally, the experimental results on simulated data and real data have demonstrated the effectiveness of the proposed algorithm.

Multiobjective neural architecture search (MONAS) methods based on evolutionary algorithms (EAs) are inefficient when the evaluation of each architecture incorporates parameter learning from scratch. A surrogate-assisted MONAS problem can be tough considering cold-start in surrogate construction, and the evaluation of predicted promising architectures could still be cumbersome. Previously solved MONAS problems are likely to convey useful knowledge that could assist solving the current MONAS problem. To take the benefit from knowledge of these previous practices, a framework tackling large-scale knowledge transfer is proposed. Through sparse-constraint transfer stacking, the surrogate for the current problem gets informative easily. With knee-region knowledge distillation from previously learned parameters of nondominated architectures, evaluation of current architectures could be efficient and credible. To avoid transferring knowledge from irrelevant problems, an iterative source selection algorithm is designed to avoid negative transfer. The proposed framework is analyzed under different source and target MONAS problem combinations. Results show that with the help of this framework, architectures with competitive performance could be found under limited evaluation budget.

Multimodal multiobjective optimization problems (MMOPs) possess multiple Pareto optimal sets (PSs) corresponding to the identical Pareto optimal front (PF). To handle MMOPs, we propose a bi-objective evolutionary algorithm (BOEA), which transforms an MMOP into a bi-objective optimization problem. This problem is constructed by the penalty boundary intersection technique and a diversity indicator to obtain multiple PSs. The first objective reflects the population convergence and factors in the population diversity in the objective space, while the other objective concentrates more on the population diversity in the decision space. Furthermore, an environmental selection strategy is designed to choose the offspring solutions, which consists of nondominated sorting based on the transformed optimization problem and hierarchical clustering for selecting promising solutions. Experiments on 34 MMOPs demonstrate that BOEA performs better than selected state-of-the-art representatives, including 22 MMOPs from CEC2019 and 12 imbalanced MMOPs. In addition, the effectiveness of BOEA is further validated by six feature selection problems in real-world applications.

Point cloud registration is an important task in computer vision, where the goal is to estimate a transformation to align a pair of point clouds. Most of the existing registration methods face the problems of poor robustness and getting stuck in local optima. Evolutionary multitasking is an effective paradigm to enhance global search capability and improve convergence characteristics through knowledge transfer across multiple related tasks. Inspired by evolutionary multitasking, this article proposes a multiform optimization approach through evolutionary multitasking for solving the point cloud registration problems. We first construct two related registration tasks with different functional landscapes to form a multiform optimization problem. Compared with methods that only focus on a single registration attribute, the two proposed tasks focus on robustness and precision of registration, respectively. Then, a new two-stage bidirectional knowledge transfer strategy is presented, which can implement efficient knowledge transfer among two related tasks. Finally, both simulations and real experiments show the power of our method. The proposed method is robust to noise, outliers, and partial overlaps and is effective in multiple real registration scenarios, such as object registration, scene reconstruction, and simultaneous localization and mapping.

Surrogate-assisted evolutionary algorithms (SAEAs) have proven to be effective in solving computationally expensive optimization problems (EOPs). However, the performance of SAEAs heavily relies on the surrogate model and infill criterion used. To improve the generalization of SAEAs and enable them to solve a wide range of EOPs, this paper proposes an SAEA called AutoSAEA, which features model and infill criterion auto-configuration. Specifically, AutoSAEA formulates model and infill criterion selection as a two-level multi-armed bandit problem (TL-MAB). The first and second levels cooperate in selecting the surrogate model and infill criterion, respectively. A two-level reward (TL-R) measures the value of the surrogate model and infill criterion, while a two-level upper confidence bound (TL-UCB) selects the model and infill criterion in an online manner. Numerous experiments validate the superiority of AutoSAEA over some state-of-the-art SAEAs on complex benchmark problems and a real-world oil reservoir production optimization problem.

When deep neural networks are used to process remote sensing data, complex network structures, and parameters limit their real-time processing deployment in satellites. Binary Neural Networks (BNNs) can overcome this obstacle by reducing network complexity. However, the binarization process naturally leads to information loss, reducing the performance of the original neural networks. In this article, three modules are introduced to improve the performance of BNNs. First, we propose an improved channel shuffle (CS) module to enhance communication between different channels in feature maps, in which the grouped pointwise convolution is used to reduce the original calculation scale. Second, the squeeze and excitation (SE) module called the attention mechanism is inserted before each layer of the binary convolution operation to recognize the importance of different feature channels. Third, the adaptive linear scaling factor (ALSF) module is proposed to make the weights of the convolution kernel more suitable for the structure of BNNs, which further improves the performance of our network. With these three modules, our CSA-Net reaches or even outperforms the other state-of-the-art (SOTA) methods with a lower computational cost. It can achieve 65.35% Top-1 accuracy on the ImageNet, 90.11% accuracy on the CIFAR10, 89.58% accuracy on the WHU-RS19, and 90.86% accuracy on the UC-Merced, respectively. Meanwhile, compared with other mainstream remote sensing classification models, the number of our parameters is reduced by two or even three orders of magnitude, and the results confirm the effectiveness of the proposed method.

Oriented detectors have become the mainstream of object detection in remote-sensing images since they provide more precise bounding boxes and contain less background. However, there remain several challenges that restrict the detection performance and need to be tackled. This article focuses on the following two aspects: 1) numerous tiny objects in remote-sensing images pose a challenge for the detectors pursuing high recall and accurate localization and 2) specific categories with large aspect ratios and arbitrary angles also trouble the regression of the detectors. We attempt to alleviate the above problems by constructing a weak feature extraction network (WFEN) and a dual appearance-aware (DA) loss. Specifically, WFEN is used to extract hierarchical weight vectors for multiscale feature layers by employing a lightweight convolutional module, aiming to fuse activation features distributed in different layers and provide pure features for subsequent regression and classification. DA loss is tailored to regressions of tiny and slender objects by dynamically modulating the associated loss on objects with various appearances, which consists of two auxiliary losses, termed scale-aware loss${\mathcal {L}}_{S}$and aspect-ratio-aware loss${\mathcal {L}}_{A}$. These two components can contribute to each other, that is, the former provides more accurate features for detection tasks, while the latter can reciprocate the former by imposing constraints on crucial objects, and together constitute an appearance sensitivity detector (ASDet). Extensive experiments on three public datasets demonstrate that our ASDet outperforms all refine-stage detectors in terms of accuracy while maintaining the superior inference speed of single-stage counterparts.

Pruning techniques have been utilized widely for convolutional neural networks (CNNs) to reduce the computation resources in remote sensing scene image classification. However, conventional pruning techniques are weight-based, which can not balance the pruning ratio and representation ability appropriately. In this paper, we propose a Data Customization-based Multiobjective Optimization Pruning (DCMOP) framework for the pruning in remote sensing scene image classification, which can not only trade-off between pruning ratio and capability for CNNs, but also speed up the evolutionary process for the pruning. We adopt the multiobjective evolutionary algorithms (MOEAs) to search for a trade-off between the pruning ratio and capability for CNNs. However, a big concern of pruning for networks via MOEAs is that the evaluation of sub-networks is time-costing. This originates that the slimmed sub-networks require a lot of retraining operation, which will burden the hardware. In order to alleviate this limitation, we design a Data Customization-based Proxy Mechanism (DCPM) to reduce the size of the input dataset in terms of the structure of the slimmed sub-network to accelerate significantly the evolutionary process for the pruning. According to this, our proposed DCMOP achieves the pruning with higher efficiency and performance by cooperating with MOEAs and DCPM. Experimental results based on four datasets of AID, NWPURESISC45, PatternNet, and WHU-RS19 show that the proposed DCMOP can achieve a balance between model performance and pruning rate, while obviously reducing the time cost of the pruning.

The powerful deep learning models in remote sensing are inseparable from the support of massive data. However, the privacy and sensitivity of remote sensing data (RSD) restrict the possibility of each party to collaboratively train and share a large general model. Although multi-party learning (MPL) is a feasible solution, it is difficult for the existing MPL methods to uniformly process different remote sensing tasks (RSTs), and the data held by each party is non-independent and identically distributed, heterogeneous and multi-sources. Therefore, it is urgent to explore a solution for the personalized processing of different RSTs. In this paper, we formulate a novel multi-party personalized collaborative learning (MPCL) framework in terms of models and tasks. Specifically, in each iteration of the communication round, we aim to decouple personalized model optimization from global model learning. Different participants are allowed to explore their personalized local models at a certain distance from the global aggregation models according to the characteristics of their local data. In terms of task personalization, MPCL provides different personalized global models to handle the corresponding RSTs. For participants with different RSTs, it can be implemented in the multi-task collaborative training strategy to explore the connection between different tasks. To demonstrate the feasibility of MPCL, we take remote sensing image classification as a case study and provide a detailed feasibility scheme. We constructed four benchmark datasets compliant with MPL and personalized MPL, including single-source and multi-source about SAR, hyperspectral and optical RSD. The experimental results demonstrate that our MPCL is superior in these four RSD, which ranked first in the competition with the classic or state-of-the-art MPL and personalized MPL algorithms. In addition, the scalability of MPCL is also verified on image segmentation RSTs of building and road extraction.

Few-shot aerial imagery segmentation refers to the task of segmenting specific objects in scenes that have not been encountered during training with a small amount of annotated data for reference. However, most existing few-shot segmentation algorithms are primarily designed for natural images, and there is still a lack of exploration in the context of remote sensing aerial imagery. In this article, we propose a novel multiscale self-attention aggregation network (MS2A2Net), dubbed MS2A2Net, to address the challenge of few-shot aerial image segmentation in terms of scarce data and network architecture. Specifically, we first incorporate the designed asymmetric momentum contrastive learning (AMCL) into the pre-training stage, to improve the representation capability of the backbone without the expensive labeled data. Then the frozen encoder is transferred to the downstream few-shot segmentation task as the feature embedding. In terms of network architecture, we design self-attention aggregation in multiscale feature fusion, to construct the dual correlation of foreground and background between support and query features at the pixel level. Besides, the coordinate attention is designed to rearrange the distribution of feature importance in both horizontal and vertical spatial order perspectives, which facilitates adaptive fusion with the multiscale features. To verify the availability of the proposed MS2A2Net, we also reconstructed two novel datasets dedicated to few-shot aerial image segmentation, called DLRSD-$4^{i}$and iSAID-$4^{i}$. The experimental results show that our approach MS2A2Net is superior in three few-shot benchmark aerial imagery segmentation datasets, which achieves competitive segmentation performance. Extensive ablation experiments also reflect the effectiveness and scalability of the proposed components and overall network architecture.

Classification of multitemporal remote sensing (MTRS) images with only a few labels of one of these images has attracted widespread interest in recent years. It usually confronts three problems: domain increment, class increment, and class disappearance. In this article, a robust self-paced incremental learning (RSPIL) is proposed to alleviate the above problems. First, change detection is used to transfer labels from the source image to the target one for formulating a combined training set. Then, the weighted classification loss and the distillation loss are considered to ensure classification performance and minimal forgetting. In particular, a novel entropy-inhibit loss is proposed to suppress the classification capability for the disappearing classes. These losses are combined with self-paced learning (SPL) by introducing a weight variable to measure the “easiness” of the training samples, which is able to automatically acquire accurate decision boundaries from easy to hard since the combined training set generated by change detection contains noisy samples and outliers. Finally, a nearest-average-eigenvector classifier and an exemplar set management strategy based on the sample weights are designed to alleviate catastrophic forgetting (CF). Twenty-four MTRS image datasets from four areas are considered in the experiments. The classification results demonstrate that the proposed method is able to alleviate CF and achieves significant improvements on multitemporal image datasets.

Recently, the field of hyperspectral image (HSI) classification has witnessed advancements with the emergence of deep learning models. Promising approaches, such as self-supervised strategies and domain adaptation, have effectively tackled the overfitting challenges posed by limited labeled samples in HSI classification. To extract comprehensive semantic information from different types of auxiliary tasks, which view the problem from multiple perspectives, and efficiently integrate multiple tasks into a single network, this paper proposes a hybrid multi-task learning framework (HyMuT) by sharing representations across multiple tasks. Based on the similarity between the data and target classification task, we construct three auxiliary tasks that are similar, related and weakly correlated to the target task, while three corresponding multi-task learning methods are integrated. The framework establishes a backbone network with a hard parameter sharing mechanism, which handles the main task and a similar spatial mask classification task. Subsequently, a hierarchical transfer multi-task learning approach is introduced to transfer the knowledge of a spatial-spectral joint mask reconstruction task from the autoencoder to the backbone network. Furthermore, a new source domain HSI dataset is introduced as an auxiliary task weakly correlated. To solve the source domain classification task and assist the hard parameter sharing mechanism, a dual adversarial classifier based on adversarial learning is employed. This classifier effectively extracts domain and task invariance. Extensive experiments are conducted on four benchmark HSI datasets to evaluate the performance. The results demonstrate that HyMuT outperforms state-of-the-art methods. This code will be available from the website: https://github.com/HaoLiu-XDU/HyMuT.

Conditioned on measurement data from sensors, the joint probabilistic data association (JPDA) filter is a popular methodology for tracking multiple objects in clutter. The JPDA filter, however, suffers from the severe track coalescence effect, i.e., tracks following objects in close proximity tend to coalesce. To improve the tracking accuracy, we propose a novel Cauchy–Schwarz set JPDA (CSSJPDA) filter by optimizing the posterior Gaussian mixture density with an iterative successive component-based optimization (ISCO) algorithm in the Cauchy–Schwarz sense. The posterior Gaussian mixture density is marginalized by Gaussian densities to estimate object states at each time step. To improve the marginalization accuracy, the posterior density is optimized in the random finite set (RFS) family. We derive a closed-form expression of the Cauchy–Schwarz divergence between the posterior density and its Gaussian approximation and use it as a cost function for density optimization. To improve the optimization efficiency, we propose the ISCO algorithm to minimize the cost function successively along one Gaussian component at a time and prove its monotone convergence. Two indicative examples are used to illustrate the effectiveness of the CSSJPDA filter. With the iteration, the cost function decreases and the Gaussian approximation becomes more accurate. Simulation results demonstrate that the CSSJPDA filter provides a good tradeoff between the tracking accuracy and the computational efficiency compared to the existing methods. Numerical experiments with a real-world dataset further verify the performances of the proposed method.

Self-supervised learning, which leverages unlabeled data to learn useful feature representations by constructing auxiliary tasks, has been widely explored in few-shot scene classification to improve the feature representation and generalization capabilities of deep models in scarce data. However, most of the current related work adopts specific self-supervised auxiliary tasks (SSATs) for combinatorial improvement, and does not explore the intrinsic connection between different pretext tasks. In practice, the linkage of SSATs is complex, and the optimization of task-sharing parameters by minimizing linear combinations of losses can be conflicting. In addition, although a single combination of SSAT can improve certain performance on the baseline, it is not the personalized optimal solution on various remote sensing datasets with diverse properties. In this article, we propose a collaborative self-supervised evolution (so-called CSENet) framework for few-shot remote sensing scene classification to automatically search for appropriate weights in balancing the task conflicts. In contrast to most existing methods, which consider all SSATs to be equally efficacious or fixed-weighted for the few-shot main task, CSENet achieves autonomous co-evolutionary optimization by encoding arbitrary self-supervised weights. Specifically, the complex self-supervised combinations for different remote sensing data are transformed into an evolutionary optimization problem, where chromosomes with weighting variables obtain the optimal combination with genetic operators. Based on the transfer learning few-shot training paradigm, CSENet first efficiently searches for optimal self-supervised combinations with potential by the proposed automatic collaborative evolution strategy and automatically adjusts the weights without manual settings. Importantly, CSENet provides both inductive and transductive inference, and supports the embedding of arbitrary SSATs. The effectiveness of the proposed framework is demonstrated by state-of-the-art (SOTA) results on three benchmark datasets.

Change detection (CD) methods have been crucial in exploring geo-environmental science. With the advancement of remote sensing (RS) technology, multimodal images acquired from different platforms and sensors are widely used for CD tasks. As an emerging task, multimodal CD (MCD) aims to achieve more comprehensive and precise detection of land cover changes through complementary information in multimodal images. However, there are significant differences between modalities, particularly in heterogeneous images. How to deal with modal differences while effectively integrating change information remains a challenge in MCD. In this article, we propose a novel adversarial feature equilibrium network (AFENet), which establishes an additional adversarial optimization to solve the equilibrium problem between modal differences and land cover changes. Our AFENet aligns the features and reduces the modal gap through a multiscale adversarial domain adaptation (MADA) approach. Meanwhile, a divergence-aware contrastive module (DCM) is designed as a regularization term for adversarial optimization. DCM affects the sensitivity of feature extractors by constraining the mutual information between changed and unchanged pixels. In this case, AFENet can maintain the consistency of feature representation while maximizing the discriminability of change targets. The features extracted from AFENet will then be integrated by our multistream feature fusion (MFF) module and utilized to generate change maps. The effectiveness of our approach is demonstrated on two scene-level multimodal RS datasets. Compared with existing methods, our AFENet achieves state-of-the-art (SOTA) performance on both datasets and outperforms the second-best$F1$score by 4.64% and 1.1%, respectively.

Anomaly detection (AD) has attracted remarkable attention in hyperspectral image (HSI) processing fields, most existing deep learning (DL) based algorithms indicate dramatic potential for detecting anomaly samples through specific training process under current scenario. However, the limited prior information and the catastrophic forgetting problem indicate crucial challenges for existing DL structure in open scenarios cross-domain detection. In order to improve the detection performance, a novel capsule differential adversarial continual learning framework (CL-CaGAN) is proposed to elevate the cross-scenario learning performance for facilitating the real application of DL-based structure in hyperspectral anomaly detection (HAD) task. First, a modified capsule structure with adversarial learning network is constructed to estimate the background distribution for surmounting the deficiency of prior information. To mitigate the catastrophic forgetting phenomenon, clustering-based sample replay strategy and a designed extra self-distillation regularization are integrated for merging the history and future knowledge in continual AD task, while the discriminative learning ability from previous detection scenario to current scenario are retained by the elaborately designed structure with continual learning strategy. In addition, the differentiable enhancement is enforced to augment the generation performance of the training data for further stabilizing the training process with better convergence, this procedure further efficiently consolidates the reconstruction ability of background samples. To verify the effectiveness of our proposed CL-CaGAN, we conduct experiments on several real HSIs, the results indicate that the proposed CL-CaGAN demonstrates higher detection performance and continuous learning capacity for mitigating the catastrophic forgetting under cross-domain scenarios.

The existing few-shot scene classification (FSSC) algorithms have achieved satisfactory results, but they are limited by the paradigm of centralized machine learning, i.e., private remote sensing data need to be centralized on a certain server for training. However, remote sensing images generally contain sensitive information such as national security and company privacy, so it is realistically difficult to collect remote sensing data from all the parties. Therefore, there is a pressing requirement in FSSC for a novel paradigm to achieve multi-party collaborative learning without compromising remote sensing data privacy. In this paper, we formulate a novel personalized multi-party few-shot learning (PMPFSL) paradigm for remote sensing scene classification. In PMPFSL, different participants can achieve multi-party collaborative learning without sacrificing the privacy of their local data, and their respective local models are able to recognize the unseen remote sensing scene categories with a small number of labeled samples. Importantly, the proposed PMPFSL is applicable to various multi-party learning algorithms and few-shot scene classification networks. Moreover, to address the problems of local model overfitting and poor discriminability of few-shot metrics, we propose the personalized adaptive distillation (PAD) scheme and multi-scale feature matching network (MSFMNet) on PMPFSL, respectively. Specifically, each participant obtains the MSFMNet with initialization parameters, and implements a certain number of local training on their respective private machines. Global aggregation is subsequently achieved by uploading only the local models to the central server. In a new round of local training, the participants realize personalized data adaptation to the global model based on the PAD. Overall, the proposed PMPFSL customizes a personalized few-shot model for each participant that is more tailored to their respective remote sensing scenarios. The experimental results demonstrate that our PMPFSL is superior in three benchmark FSSC datasets. We also extensively studied and analyzed the contributions of PAD and MSFMNet in the proposed PMPFSL framework.

Hyperspectral image (HSI) inpainting is a fundamental task in remote sensing image processing, which is helpful for subsequent applications such as classification and unmixing. Recently, tensor-train decomposition (TTD)-based low-rank methods have achieved great success in image inpainting because of the balanced tensor unfolding and the use of tensor augmentation (TA). However, the TTD algorithm only performs TA on the third mode, and cannot effectively mine the spectral domain information of HSIs. Aiming at this problem, this article extends TA to each mode of the tensor and proposes a full-mode-augmentation TT decomposition (FTTD). More precisely, the HSI is cast along each mode to get a 3-D$N$th-order tensor sequence, and all the tensors in the sequence are performed TTD to get$3N$factor tensors that model the correlation on various modes. Then, the full-mode-augmentation tensor unfolding method is given by performing TT unfolding on each$N$th-order tensor. We implement FTTD by minimizing the rank of the unfolding matrix and thus provide the tensor completion framework based on full-mode-augmentation tensor-train rank minimization. Finally, using the framework, we optimize the current two classical iterative algorithms nuclear norm minimization and parallel matrix decomposition. These two algorithms utilize the unfolding matrices in the framework to effectively mine spatial and spectral information, thereby making the restored tensor more approximate to the original HSI. Experiments on various HSIs have shown that the proposed method outperforms compared methods in terms of visual and quantitative measures.

Remote sensing scene classification (RSSC) aims to understand and analyze the semantic information at the scene level with complex geographical properties. Despite the profound success of advanced deep models in automatically capturing hierarchical embedding representations and the gradual dominant trend in RSSC, it still remains a great challenge to precisely focus on targets at variable scales that are considered highly relevant to the corresponding scene and separated from the background. Motivated by this recognition, in this article, we present the gradient-guided multiscale focal attention network (GMFANet) for RSSC to adaptively localize the representative multiscale semantic representation for complex scenes. In particular, a lightweight parameterized hierarchical multiscale attention (HMA) mechanism is proposed, which constitutes the main aim of adaptively enhancing physical detail and high-level semantic information at different layers, rather than regarding each scale set with equivalent insight, while eliminating redundant information inherent in conventional attention mechanisms. Subsequently, a gradient-guided spatial focused attention (GSFA) module is specifically designed to accurately localize critical regions at multiple scales, with the dynamic combination of gradient-activated reference attention map and prediction attention map from supervised information-based learning. In addition, a curriculum-driven dynamic attention fusion (CDAF) strategy is tailored to fuse the spatial attention above from easy to hard for avoiding from poor local optimum and decreasing the early learning ambiguity. Our extensive comparative experiments and ablation analyses implemented on real-world public RSSC datasets indicate that our approach achieves the state-of-the-art performance exactly. The code is available athttps://github.com/bling2beyond/GMFANet.

Change detection (CD) in multispectral remote sensing (RS) imagery suffers from low spectral resolution which can lead to degraded recognition of change information from land cover objects. Considering that natural hyperspectral imagery (HSI) is much higher in spectral resolution and more accessible, using it to enhance the spectral information of RS multispectral imagery for CD can improve performance. To achieve this, we propose a spectral knowledge transfer (SKT) framework to allow the creation of pseudo-hyperspectral RS images from the available RS multispectral ones without the need for the real pairs of RS multispectral and hyperspectral images, typically required by existing RS spectral enhancement methods. Specifically, an autoencoder is first trained based on the available pairs of natural HSI and its multispectral counterparts and then calibrated via the available RS multispectral images. The finally obtained decoder module is used to generate the pseudo-hyperspectral image from an input RS multispectral image. We further propose a multispectrum collaborative CD (MCCD) framework that leverages both the real multispectral images and the pseudo hyperspectral images generated from them in a collaborative way to achieve performance improvement. Extensive experiments on two large-scale RS CD datasets and eight existing deep learning-based CD methods demonstrate the stronger efficacy of the proposed method.

Federated learning has gained prominence as an effective solution for addressing data silos, enabling collaboration among multiple parties without sharing their data. However, existing federated learning algorithms often neglect the challenge posed by multi-modal data distribution. Moreover, previous pioneering work face limitations in encrypting the exponential and logarithmic operations of the objective function with multiple independent variables, and they rely on a third-party cooperator for encryption. To address these limitations, this paper introduces a universal multi-modal vertical federated learning framework. To tackle the data distribution challenge, we propose a two-step multi-modal transformer model that captures cross-domain semantic features effectively. For encryption, where traditional additively homomorphic encryption algorithms fall short by supporting only addition and multiplication, we employ bivariate Taylor series expansion to transform the objective function. Integrating these components, we present a comprehensive training and transmission protocol that eliminates the need for a third-party cooperator during the encryption process. Extensive experiments conducted on diverse video-text and image-text datasets validate the superior performance of our framework compared to state-of-the-art approaches, affirming its effectiveness in multi-modal vertical federated learning settings.

Multi-layer networks treat various types of interactions at each level to model complex systems in nature and society, and clustering of them is of great significance for revealing mechanisms of systems. Vast majority of current algorithms focus on identifying the common communities in clear multi-layer networks, and few attempt has been devoted to the detection of layer-specific communities in noised ones. To address these issues, a joint learning algorithm withGraphDenoising andStructureLearning (calledGDSL) for the detection of layer-specific communities in noised multi-layer networks is proposed, which simultaneously integrates graph denoising, structure learning, and module detection. To remove noise of networks, GDSL re-constructs affinity graphs for the original ones by preserving community structure. To enhance robustness and discriminative of features, GDSL explores the relations of features among various layers with the Hilbert-Schmidt Independence Criterion and structure learning. Finally, GDSL joins all these procedures with an objective function, and deduces optimization rules. The results show that GDSL not only significantly outperforms baselines but also enhances the robustness of the algorithm, providing an effective model for community detection in noised multi-layer networks.

Masked autoencoder (MAE) is a recently widely used self-supervised learning method that has achieved great success in NLP and computer vision. However, the potential advantages of masked pre-training for point cloud understanding have not been fully explored. There is preliminary work on MAE-based point clouds using the Transformer architecture to explore low-level geometric representations in 3D space, which is insufficient for fine-grained decoding completion and downstream tasks. Inspired by multimodality, we propose Inter-MAE, a inter-modal MAE method for self-supervised learning on point clouds. Specifically, we first use Point-MAE as a baseline to partition point clouds into random low percentage of visible and high percentage of masked point patches. Then, a standard Transformer-based autoencoder is built by asymmetric design and shifting mask operations, and latent features are learned from the visible point patches aiming to recover the masked point patches. In addition, we generate image features based on ViT after point cloud rendering to form inter-modal contrastive learning with the decoded features of the completed point patches. Extensive experiments show that the proposed Inter-MAE generates pre-trained models that are effective and exhibit superior results in various downstream tasks. For example, an accuracy of 85.4% is achieved on ScanObjectNN and 86.3% on ShapeNetPart, outperforming other state-of-the-art self-supervised learning methods. Notably, our work establishes for the first time the feasibility of applying image modality to masked point clouds.

Learning effective representations from unlabeled data is a challenging task for point cloud understanding. As the human visual system can map concepts learned from 2D images to the 3D world, and inspired by recent multimodal research, we introduce data from point cloud modality and image modality for joint learning. Based on the properties of point clouds and images, we propose CrossNet, a comprehensive intra- and cross-modal contrastive learning method that learns 3D point cloud representations. The proposed method achieves 3D-3D and 3D-2D correspondences of objectives by maximizing the consistency of point clouds and their augmented versions, and with the corresponding rendered images in invariant space. We further distinguish the rendered images into RGB and grayscale images to extract color and geometric features, respectively. These training objectives combine feature correspondences between modalities to combine rich learning signals from point clouds and images. Our CrossNet is simple: we add a feature extraction module and a projection head module to the point cloud and image branches, respectively, to train the backbone network in a self-supervised manner. After the network is pretrained, only the point cloud feature extraction module is required for fine-tuning and directly predicting results for downstream tasks. Our experiments on multiple benchmarks demonstrate improved point cloud classification and segmentation results, and the learned representations can be generalized across domains.

The self-attention (SA) network revisits the essence of data and has achieved remarkable results in text processing and image analysis. SA is conceptualized as a set operator that is insensitive to the order and number of data, making it suitable for point sets embedded in 3D space. However, working with point clouds still poses challenges. To tackle the issue of exponential growth in complexity and singularity induced by the original SA network without position encoding, we modify the attention mechanism by incorporating position encoding to make it linear, thus reducing its computational cost and memory usage and making it more feasible for point clouds. This article presents a new framework called multiscale point cloud transformer (MPCT), which improves upon prior methods in cross-domain applications. The utilization of multiple embeddings enables the complete capture of the remote and local contextual connections within point clouds, as determined by our proposed attention mechanism. Additionally, we use a residual network to facilitate the fusion of multiscale features, allowing MPCT to better comprehend the representations of point clouds at each stage of attention. Experiments conducted on several datasets demonstrate that MPCT outperforms the existing methods, such as achieving accuracies of 94.2% and 84.9% in classification tasks implemented on ModelNet40 and ScanObjectNN, respectively.

The pandemic of coronavirus disease 2019 (COVID-19) has led to a global public health crisis, which caused millions of deaths and billions of infections, greatly increasing the pressure on medical resources. With the continuous emergence of viral mutations, developing automated tools for COVID-19 diagnosis is highly desired to assist the clinical diagnosis and reduce the tedious workload of image interpretation. However, medical images in a single site are usually of a limited amount or weakly labeled, while integrating data scattered around different institutions to build effective models is not allowed due to data policy restrictions. In this article, we propose a novel privacy-preserving cross-site framework for COVID-19 diagnosis with multimodal data, seeking to effectively leverage heterogeneous data from multiple parties while preserving patients' privacy. Specifically, a Siamese branched network is introduced as the backbone to capture inherent relationships across heterogeneous samples. The redesigned network is capable of handling semisupervised inputs in multimodalities and conducting task-specific training, in order to improve the model performance of various scenarios. The framework achieves significant improvement compared with state-of-the-art methods, as we demonstrate through extensive simulations on real-world datasets.

Three-dimensional point cloud registration is an important field in computer vision. Recently, due to the increasingly complex scenes and incomplete observations, many partial-overlap registration methods based on overlap estimation have been proposed. These methods heavily rely on the extracted overlapping regions with their performances greatly degraded when the overlapping region extraction underperforms. To solve this problem, we propose a partial-to-partial registration network (RORNet) to find reliable overlapping representations from the partially overlapping point clouds and use these representations for registration. The idea is to select a small number of key points called reliable overlapping representations from the estimated overlapping points, reducing the side effect of overlap estimation errors on registration. Although it may filter out some inliers, the inclusion of outliers has a much bigger influence than the omission of inliers on the registration task. The RORNet is composed of overlapping points' estimation module and representations' generation module. Different from the previous methods of direct registration after extraction of overlapping areas, RORNet adds the step of extracting reliable representations before registration, where the proposed similarity matrix downsampling method is used to filter out the points with low similarity and retain reliable representations, and thus reduce the side effects of overlap estimation errors on the registration. Besides, compared with previous similarity-based and score-based overlap estimation methods, we use the dual-branch structure to combine the benefits of both, which is less sensitive to noise. We perform overlap estimation experiments and registration experiments on the ModelNet40 dataset, outdoor large scene dataset KITTI, and natural data Stanford Bunny dataset. The experimental results demonstrate that our method is superior to other partial registration methods. Our code is available at https://github.com/superYuezhang/RORNet.

Multiparty learning (MPL) is an emerging framework for privacy-preserving collaborative learning. It enables individual devices to build a knowledge-shared model and remaining sensitive data locally. However, with the continuous increase of users, the heterogeneity gap between data and equipment becomes wider, which leads to the problem of model heterogeneous. In this article, we concentrate on two practical issues: data heterogeneous problem and model heterogeneous problem, and propose a novel personal MPL method named device-performance-driven heterogeneous MPL (HMPL). First, facing the data heterogeneous problem, we focus on the problem of various devices holding arbitrary data sizes. We introduce a heterogeneous feature-map integration method to adaptively unify the various feature maps. Meanwhile, to handle the model heterogeneous problem, as it is essential to customize models for adapting to the various computing performances, we propose a layer-wise model generation and aggregation strategy. The method can generate customized models based on the device's performance. In the aggregation process, the shared model parameters are updated through the rules that the network layers with the same semantics are aggregated with each other. Extensive experiments are conducted on four popular datasets, and the result demonstrates that our proposed framework outperforms the state of the art (SOTA).

With the rapid development of remote sensing (RS) technology, high-resolution RS image change detection (CD) has been widely used in many applications. Pixel-based CD techniques are maneuverable and widely used, but vulnerable to noise interference. Object-based CD techniques can effectively utilize the abundant spectrum, texture, shape, and spatial information but easy-to-ignore details of RS images. How to combine the advantages of pixel-based methods and object-based methods remains a challenging problem. Besides, although supervised methods have the capability to learn from data, the true labels representing changed information of RS images are often hard to obtain. To address these issues, this article proposes a novel semisupervised CD framework for high-resolution RS images, which employs small amounts of true labeled data and a lot of unlabeled data to train the CD network. A bihierarchical feature aggregation and extraction network (BFAEN) is designed to achieve the pixelwise together with objectwise feature concatenation feature representation for the comprehensive utilization of the two-level features. In order to alleviate the coarseness and insufficiency of labeled samples, a confident learning algorithm is used to eliminate noisy labels and a novel loss function is designed for training the model using true- and pseudo-labels in a semisupervised fashion. Experimental results on real datasets demonstrate the effectiveness and superiority of the proposed method.

It is attractive to extract plausible 3-D information from a single 2-D image, and self-supervised learning has shown impressive potential in this field. However, when only monocular videos are available as training data, moving objects at similar speeds to the camera can disturb the reprojection process during training. Existing methods filter out some moving pixels by comparing pixelwise photometric error, but the illumination inconsistency between frames leads to incomplete filtering. In addition, existing methods calculate photometric error within local windows, which leads to the fact that even if an anomalous pixel is masked out, it can still implicitly disturb the reprojection process, as long as it is in the local neighborhood of a nonanomalous pixel. Moreover, the ill-posed nature of monocular depth estimation makes the same scene correspond to multiple plausible depth maps, which damages the robustness of the model. In order to alleviate the above problems, we propose: 1) a self-reprojection mask to further filter out moving objects while avoiding illumination inconsistency; 2) a self-statistical mask method to prevent the filtered anomalous pixels from implicitly disturbing the reprojection; and 3) a self-distillation augmentation consistency loss to reduce the impact of ill-posed nature of monocular depth estimation. Our method shows superior performance on the KITTI dataset, especially when evaluating only the depth of potential moving objects.

We explore the effect of geometric structure descriptors on extracting reliable correspondences and obtaining accurate registration for point cloud registration. The point cloud registration task involves the estimation of rigid transformation motion in unorganized point cloud, hence it is crucial to capture the contextual features of the geometric structure in point cloud. Recent coordinates-only methods ignore numerous geometric information in the point cloud which weaken ability to express the global context. We propose Enhanced Geometric Structure Transformer to learn enhanced contextual features of the geometric structure in point cloud and model the structure consistency between point clouds for extracting reliable correspondences, which encodes three explicit enhanced geometric structures and provides significant cues for point cloud registration. More importantly, we report empirical results that Enhanced Geometric Structure Transformer can learn meaningful geometric structure features using none of the following: (i) explicit positional embeddings, (ii) additional feature exchange module such as cross-attention, which can simplify network structure compared with plain Transformer. Extensive experiments on the synthetic dataset and real-world datasets illustrate that our method can achieve competitive results.

Two-dimensional (2D) feature extraction techniques are specifically designed for reducing the dimension of data in matrix representation. Existing methods mostly rely on bilateral projections of matrices. This rasterized manner critically limits the freedom of feature combinations, and thus degrades the fitting ability of models. The robustness of previous techniques is also unsatisfactory due to the insufficiency of discerning outliers with severe damages. In this work, we propose a novel bilinear subspace learning model to achieve flexible and robust two-dimensional feature extraction. The features are exploited in a multilayer bilinear low-rank space under a collaborative orthogonality constraint, and a self-sparse robust strategy is imposed. The model greatly extends the projection space and relaxes the restriction in conventional orthogonal space. We accordingly design a tactful and efficient optimization method based on the coordinate descent method, optimally addressing the proposed model. Experimental results demonstrate the excited improvements of our extracted features in image classification task.

Existing work shows that 3D point clouds produce only about a 4% drop in semantic segmentation even at 1% random point annotation, which inspires us to further explore how to achieve better results at lower cost. As scene point clouds provide position and color information and often used in tandem as the only input, with little work going into segmentation by fusing information from dual spaces. To optimize point cloud representations, we propose a novel framework for the dual representation query network (DRQNet). The proposed framework partitions the input point cloud into position and color spaces, using the separately extracted geometric structure and semantic context to create an internal supervisory mechanism that bridges the dual spaces and fuses the information. Adopting sparsely annotated points as the query set, DRQNet provide guidance and perceptual information for multi-stage point clouds through random sampling. More, to differentiate and enhance the features generated by local neighbourhoods within multiple perceptual fields, we design a representation selection module to identify the contributions made by the position and color of each query point, and weight them adaptively according to reliability. The proposed DRQNet is robust to point cloud analysis and eliminates the effects of irregularities and disorder. Our method achieves significant performance gains on three mainstream benchmarks.

Privacy-preserving collaborative learning enables resource-constrained edge devices (e.g., Internet of Things (IoT) devices and smartphones) to build a knowledge-shared model while keeping individual data locally, achieving privacy preservation by designing an effective communication protocol. However, the learning paradigm raises high requirements for aligned input features of models, which is hard to realize in complicated IoT scenarios with various monitoring indicators. In this article, we propose a novel collaborative learning framework that is tolerant of IoT devices with unaligned feature spaces. Local bilevel optimizations for both model parameters and input features are performed iteratively in the training phase, in which the internal correlations of local sensor data provide additional guidance for the feature inference and completion. The scheme breaks hardware boundaries among various IoT devices in collaboration with the assistance of model inversion inference, which gains a new perspective on the utilization of model confidentiality and requires minimal modifications to the existing collaborative learning process. The framework achieves significant improvement compared with state-of-the-art methods, as we demonstrate through extensive simulations on real-world data sets.

Deep learning (DL) have recently achieved outstanding performance in change detection of multitemporal images. However, most existing DL-based change detection methods still suffer from the problem of insufficient labeled training samples. To overcome this limitation, an unsupervised superpixel-guided self-supervised learning network (S3Net) is proposed for detecting changes occurred on the land surface. By performing principal component analysis on two input images, a triple-channel pseudo-color image containing the main information of both images is first generated, which is used for superpixel segmentation to produce homogeneous image objects. Then, a siamese network composing of two identical subnetworks with shared weight based on transfer learning is trained for pretext task in a self-supervised learning way, aiming to obtain multiscale object-level spatial feature difference images. On this basis, a high-quality difference image is generated by incorporating the pixel-level and object-level difference information using a simple weighted fusion strategy, which can be analyzed by thresholding to produce the final binary change map. The experimental results on four real-world datasets from different sensors show that the proposed approach can obtain superior performance in comparison with several state-of-the-art change detection methods, which further demonstrates its effectiveness and practicability. We make our data and code publicly available (https://github.com/OMEGA-RS/S3Net_CD).

Multi-party learning provides an effective approach for training a machine learning model, e.g., deep neural networks (DNNs), over decentralized data by leveraging multiple decentralized computing devices, subjected to legal and practical constraints. Different parties, so-called local participants, usually provide heterogenous data in a decentralized mode, leading to non-IID data distributions across different local participants which pose a notorious challenge for multi-party learning. To address this challenge, we propose a novel heterogeneous differentiable sampling (HDS) framework. Inspired by the dropout strategy in DNNs, a data-driven network sampling strategy is devised in the HDS framework, with differentiable sampling rates which allow each local participant to extract from a common global model the optimal local model that best adapts to its own data properties so that the size of the local model can be significantly reduced to enable more efficient inference. Meanwhile, co-adaptation of the global model via learning such local models allows for achieving better learning performance under non-IID data distributions and speeds up the convergence of the global model. Experiments have demonstrated the superiority of the proposed method over several popular multi-party learning techniques in the multi-party settings with non-IID data distributions.

Rigid registration is a transformation estimation problem between two point clouds. The two point clouds captured may partially overlap owing to different viewpoints and acquisition times. Some previous correspondence matching based methods utilize an encoder-decoder network to carry out partial-to-partial registration task and adopt a skip-connection structure to convey information between the encoder and decoder. However, equally revisiting them with skip-connection may introduce the information redundancy, and limit the feature learning ability of the entire network. To address these problems, we propose a skip-attention based correspondence filtering network (SACF-Net) for point cloud registration. A novel feature interaction mechanism is designed to utilize both low-level geometric information and high-level context-aware information to enhance the original pointwise matching map. Additionally, a skip-attention based correspondence filtering method is proposed to selectively revisits features in the encoder at different resolutions, allowing the decoder to extract high-quality correspondences within overlapping regions. We conduct comprehensive experiments on indoor and outdoor scene datasets, and the results show that the proposed SACF-Net yields unprecedented performance improvements.

Point cloud registration is a key problem in the application of computer vision to robotics, autopilot and other fields. However, because the object is partially covered up or the resolution of 3D scanners is different, point clouds collected by the same sense may be inconsistent and even incomplete. Inspired by the recently proposed learning-based approaches, we propose Inliers Estimation Network (INENet) which includes a self-designed threshold prediction network and a probability estimation network with adaptive similarity mutual attention to help to find the overlapping area of the point clouds. In order to solve the above problems, we divide the partially overlapping point cloud registration task into two sub-tasks: overlapping areas detection and registration. The threshold prediction network can automatically calculate the threshold according to the input point clouds, and then the probability estimation network estimates the overlapping points by using threshold. The advantages of the proposed approach include: (1) threshold prediction network avoids bias and the complexity of manually adjusting the threshold. (2) Probability estimation network with similarity matrix can deeply fuse the information between a pair of point clouds, which is helpful to improve the accuracy. (3) INENet can be easily integrated into other overlapping region sensitive algorithms and without adjusting parameters. We conduct experiments on the ModelNet40, S3DIS and 3DMatch data sets. Specifically, the rotation error of the registration algorithm integrated with INENet is improved by at least 25% compared with direct partial overlap registration, our method improves the$F_{1} $score by 5% and has better anti-noise ability compared with the existing overlap detection methods, showing the effectiveness of the proposed method.

Non-negative matrix factorization (NMF) is one of the most popular techniques for data representation and clustering and has been widely used in machine learning and data analysis. NMF concentrates the features of each sample into a vector and approximates it by the linear combination of basis vectors, such that the low-dimensional representations are achieved. However, in real-world applications, the features usually have different importance. To exploit the discriminative features, some methods project the samples into the subspace with a transformation matrix, which disturbs the original feature attributes and neglects the diversity of samples. To alleviate the above problems, we propose the feature weighted NMF (FNMF) in this article. The salient properties of FNMF can be summarized as three-fold: 1) it learns the weights of features adaptively according to their importance; 2) it utilizes multiple feature weighting components to preserve the diversity; and 3) it can be solved efficiently with the suggested optimization algorithm. The performance on synthetic and real-world datasets demonstrates that the proposed method obtains the state-of-the-art performance.

The performance of machine learning algorithms heavily relies on the availability of a large amount of training data. However, in reality, data usually reside in distributed parties such as different institutions and may not be directly gathered and integrated due to various data policy constraints. As a result, some parties may suffer from insufficient data available for training machine learning models. In this article, we propose a multiparty dual learning (MPDL) framework to alleviate the problem of limited data with poor quality in an isolated party. Since the knowledge-sharing processes for multiple parties always emerge in dual forms, we show that dual learning is naturally suitable to handle the challenge of missing data, and explicitly exploits the probabilistic correlation and structural relationship between dual tasks to regularize the training process. We introduce a feature-oriented differential privacy with mathematical proof, in order to avoid possible privacy leakage of raw features in the dual inference process. The approach requires minimal modifications to the existing multiparty learning structure, and each party can build flexible and powerful models separately, whose accuracy is no less than nondistributed self-learning approaches. The MPDL framework achieves significant improvement compared with state-of-the-art multiparty learning methods, as we demonstrated through simulations on real-world datasets.

An expensive multimodal optimization problem (EMMOP) is that the computation of the objective function is time consuming and it has multiple global optima. This article proposes a decomposition differential evolution (DE) based on the radial basis function (RBF) for EMMOPs, called D/REM. It mainly consists of two phases: the promising subregions detection (PSD) and the local search phase (LSP). In PSD, a population update strategy is designed and the mean-shift clustering is employed to predict the promising subregions of EMMOP. In LSP, a local RBF surrogate model is constructed for each promising subregion and each local RBF surrogate model tracks a global optimum of EMMOP. In this way, an EMMOP is decomposed into many expensive global optimization subproblems. To handle these subproblems, a popular DE variant, JADE, acts as the search engine to deal with these subproblems. A large number of numerical experiments unambiguously validate that D/REM can solve EMMOPs effectively and efficiently.

Temporal networks are ubiquitous in nature and society, and tracking the dynamics of networks is fundamental for investigating the mechanisms of systems. Dynamic communities in temporal networks simultaneously reflect the topology of the current snapshot (clustering accuracy) and historical ones (clustering drift). Current algorithms are criticized for their inability to characterize the dynamics of networks at the vertex level, independence of feature extraction and clustering, and high time complexity. In this study, we solve these problems by proposing a novel joint learning model for dynamic community detection in temporal networks (also known as jLMDC) via joining feature extraction and clustering. This model is formulated as a constrained optimization problem. Vertices are classified into dynamic and static groups by exploring the topological structure of temporal networks to fully exploit their dynamics at each time step. Then, jLMDC updates the features of dynamic vertices by preserving features of static ones during optimization. The advantage of jLMDC is that features are extracted under the guidance of clustering, promoting performance, and saving the running time of the algorithm. Finally, we extend jLMDC to detect the overlapping dynamic community in temporal networks. The experimental results on 11 temporal networks demonstrate that jLMDC improves accuracy up to 8.23% and saves 24.89% of running time on average compared to state-of-the-art methods.

Cardinality constraint, namely, constraining the number of nonzero outputs of models, has been widely used in structural learning. It can be used for modeling the dependencies between multidimensional labels. In hashing, the final outputs are also binary codes, which are similar to multidimensional labels. It has been validated that estimating how many 1's in a multidimensional label vector is easier than directly predicting which elements are 1 and estimating cardinality as a prior step will improve the classification performance. Hence, in this article, we incorporate cardinality constraint into the unsupervised image hashing problem. The proposed model is divided into two steps: 1) estimating the cardinalities of hashing codes and 2) then estimating which bits are 1. Unlike multidimensional labels that are known and fixed in the training phase, the hashing codes are generally learned through an iterative method and, therefore, their cardinalities are unknown and not fixed during the learning procedure. We use a neural network as a cardinality predictor and its parameters are jointly learned with the hashing code generator, which is an autoencoder in our model. The experiments demonstrate the efficiency of our proposed method.

Complex systems in nature and society consist of various types of interactions, where each type of interaction belongs to a layer, resulting in the so-called multilayer networks. Identifying specific modules for each layer is of great significance for revealing the structure-function relations in multilayer networks. However, the available approaches are criticized undesirable because they fail to explicitly the specificity of modules, and balance the specificity and connectivity of modules. To overcome these drawbacks, we propose an accurate and flexible algorithm by joint learning matrix factorization and sparse representation (jMFSR) for specific modules in multilayer networks, where matrix factorization extracts features of vertices and sparse representation discovers specific modules. To exploit the discriminative latent features of vertices in multilayer networks, jMFSR incorporates linear discriminant analysis (LDA) into non-negative matrix factorization (NMF) to learn features of vertices that distinguish the categories. To explicitly measure the specificity of features, jMFSR decomposes features of vertices into common and specific parts, thereby enhancing the quality of features. Then, jMFSR jointly learns feature extraction, common-specific feature factorization, and clustering of multilayer networks. The experiments on 11 datasets indicate that jMFSR significantly outperforms state-of-the-art baselines in terms of various measurements.

Graph classification aims to predict the label associated with a graph and is an important graph analytic task with widespread applications. Recently, graph neural networks (GNNs) have achieved state-of-the-art results on purely supervised graph classification by virtue of the powerful representation ability of neural networks. However, almost all of them ignore the fact that graph classification usually lacks reasonably sufficient labeled data in practical scenarios due to the inherent labeling difficulty caused by the high complexity of graph data. The existing semisupervised GNNs typically focus on the task of node classification and are incapable to deal with graph classification. To tackle the challenging but practically useful scenario, we propose a novel and general semisupervised GNN framework for graph classification, which takes full advantage of a slight amount of labeled graphs and abundant unlabeled graph data. In our framework, we train two GNNs as complementary views for collaboratively learning high-quality classifiers using both labeled and unlabeled graphs. To further exploit the view itself, we constantly select pseudo-labeled graph examples with high confidence from its own view for enlarging the labeled graph dataset and enhancing predictions on graphs. Furthermore, the proposed framework is investigated on two specific implementation regimes with a few labeled graphs and the extremely few labeled graphs, respectively. Extensive experimental results demonstrate the effectiveness of our proposed semisupervised GNN framework for graph classification on several benchmark datasets.

The passive localization system (PLS) is fundamental to many wireless applications. The deployment of the monitoring stations plays a key role in the performance of the PLSes. However, the workflow of the emerging cutting-edge PLSes is becoming more flexible in the complicated environment, which makes it hard to optimize the deployment. To fulfill the requirement of the real-world applications, we propose a multiobjective PLS deployment optimization model, including a surrogate geometric dilution of precision (S-GDOP) model and a system coverage indicator to meet the demand for the detection performance of the known and unknown targets. The proposed S-GDOP is separable and open to various performance-related factors in this article. Motivated by the various cooperation mechanisms and the empirical deployment patterns, we propose a bilevel gene-based multiobjective memetic algorithm within the decomposition framework to solve this problem. By maintaining an adaptive multicomponent gene population (MCGP) and a local pivot (LP)-based local search, the population evolves on two precise and consecutive gene levels, which effectively utilizes the problem and evolution-related heuristic information. The proposed algorithm outperforms another four popular algorithms in 83.3% bilateral comparisons and obtains more implicit deployment patterns, clearer deployment structures, and better converged Pareto fronts.

End-to-end deep clustering method utilizes deep neural networks to jointly learn representation features and clustering assignments. Although many k-means-friendly deep clustering models have been explored, the existing division-based methods tend to directly implement clustering with a specific number of clusters, which suffers from poor performance resulted from indistinguishable clusters, and contributes to bad local optimum. At the same time, the representation learning of fuzzy$c$-means clustering in the feature space still needs more research. In this article, a deep fuzzy curriculum clustering method with the learning strategy of clustering from easy to complex automatically is proposed to tackle the above issues. First, considering the soft flexible allocation of fuzzy$c$-means and the preservation of local structure of original data, the fuzzy clustering loss and the autoencoder's reconstruction loss are constructed to learn the embedded features and clustering centers simultaneously. Second, curriculum loss is introduced into the constraint to make clusters successively merge in line with implementing clustering from easy to complex, and realize the bottom-up deep aggregative clustering automatically. In addition, novel curriculum information is proposed as constraint to guide the merging of clusters belonging to the same class. Experimental results on four real-world datasets show the superiority of the proposal.

Change detection (CD) in synthetic aperture radar (SAR) images aims to detect changed areas by considering the changes in backscattering coefficients. However, the changes can be further divided into positive and negative changes in terms of the increase or decrease of backscattering coefficient, so the CD task can be divided into binary and ternary according to the number of existent categories. This paper introduces an M-nary (binary or ternary) SAR change detection procedure based on the generative adversarial network (GAN) and neural architecture search (NAS) strategy to detect which changes exist in the SAR image-pair and design specialized classifiers for both binary and ternary CD. First, a difference image generation approach based on the salient changed region extraction and neighborhood information is designed for a robust difference representation on the M-nary CD. Due to the further subdivision of changes, the insufficiency of labeled data presents the M-nary change detection with a dilemma. Concerning the lack of labeled information, this paper presents a labeled sample generation strategy based on the GAN architecture search to supplement sample data. Since GAN training is inherently unstable, NAS provides an effective means of searching GAN architecture automatically and ameliorates the reliability of generated samples. During the architecture search procedure, a double-phase evolutionary search strategy is introduced to further improve the stability of GAN training. The experimental results with theoretical analysis prove the validity, robustness, and potential of our method in synthetic as well as real SAR datasets.

Classification of remote sensing image series which differ in quality and details, has impportant implications for the analysis of land cover, whereas it is expensive and time-consuming as a result of manual annotations. Fortunately, domain adaptation (DA) provides an outstanding solution to the problem. However, information loss while aligning two distributions often exists in traditional DA methods, which impacts the effect of classification with DA. To alleviate this issue, an inter-domain representation alignment and fine-tuning based network (RAFNet) is proposed for image series classification. Inter-domain representation alignment, which is fulfilled by a variational auto-encoder (VAE) trained by both source and target data, encourages reducing the discrepancy between the two marginal distributions of different domains and simultaneously preserving more data properties. As a result, RAFNet, which fuses the multi-scale aligned representations, performs classification task in the target domain after well trained with supervised learning in the source domain. Specifically, the multi-scale aligned representations of RAFNet is acquired by duplicating the frozen encoder of VAE. Then, an information based loss function is designed to fine-tune RAFNet, in which both the unchanged and changed information implied in change maps is completely used to learn the discriminative features better and make the model more generalized for the target domain. Finally, experiment studies on three datasets validate the effectiveness of RAFNet with considerable segmentation accuracy even the target data has no access to any annotated information.

Self-supervised contrastive learning (CL) can learn high-quality feature representations that are beneficial to downstream tasks without labeled data. However, most CL methods are for image-level tasks. For the fine-grained change detection (FCD) tasks, such as change or change trend detection of some specific ground objects, it is usually necessary to perform pixel-level discriminative analysis. Therefore, feature representations learned by image-level CL may have limited effects on FCD. To address this problem, we propose a self-supervised global–local contrastive learning (GLCL) framework, which extends the instance discrimination task to the pixel level. GLCL follows the current mainstream CL paradigm and consists of four parts, including data augmentation to generate different views of the input, an encoder network for feature extraction, a global CL head, and a local CL head to perform image-level and pixel-level instance discrimination tasks, respectively. Through GLCL, features belonging to different perspectives of the same instance will be pulled closer, while features of different instances will be alienated, which can enhance the discriminativeness of feature representations from both global and local perspectives, thereby facilitating downstream FCD tasks. In addition, GLCL makes a targeted structural adaptation to FCD, i.e., the encoder network is undertaken by the common backbone networks of FCD, which can accelerate the deployment on downstream FCD tasks. Experimental results on several real datasets show that compared with other parameter initialization methods, the FCD models pretrained by GLCL can obtain better detection performance.

Deep convolutional neural networks have achieved much success in remote sensing image change detection (CD) but still suffer from two main problems. First, existing multi-scale feature fusion methods often employ redundant feature extraction and fusion strategies, which often leads to high computational costs and memory usage. Second, the regular attention mechanism in CD is difficult to model spatial-spectral features and generate 3D attention weights at the same time, ignoring the cooperation between spatial features and spectral features. To address the above issues, an efficient ultra-lightweight spatial-spectral feature cooperation network (USSFC-Net) is proposed for CD in this paper. The proposed USSFC-Net has two main advantages. First, a multi-scale decoupled convolution (MSDConv) is designed, which is clearly different from the popular atrous spatial pyramid pooling (ASPP) module and its variants since it can flexibly capture the multi-scale features of changed objects by using cyclic multi-scale convolution. Meanwhile, the design of MSDConv can greatly reduce the number of parameters and computational redundancy. Second, an efficient spatial-spectral feature cooperation strategy (SSFC) is introduced to obtain richer features. The SSFC differs from existing 2D attention mechanisms since it learns 3D spatial-spectral attention weights without adding any parameters. The experiments on three datasets for remote sensing image CD demonstrate that the proposed USSFC-Net achieves better CD accuracy than most convolutional neural networks-based methods and requires lower computational costs and fewer parameters, even it is superior to some Transformer-based methods. The code is available at https://github.com/SUST-reynole/USSFC-Net.

Self-supervised learning is an effective way to solve model collapse for few-shot remote sensing scene classification (FSRSSC). However, most self-supervised contrastive learning auxiliary tasks perform poorly on the high interclass similarity problem in FSRSSC. Furthermore, it is time-consuming and computationally expensive to obtain the best combination among numerous self-supervised auxiliary tasks. In practical applications, we may encounter difficulties in remote sensing data acquisition and labeling, while most FSRSSC studies only focus on the former. To alleviate the above problems, we propose a multiform ensemble self-supervised learning (MES2L) framework for FSRSSC in this article. Based on the transfer learning-based few-shot scheme, we design a novel global–local contrastive learning auxiliary task to solve the low interclass separability problem. The self-attention mechanism is designed in the local contrast features to investigate the intrinsic associations between different remote sensing scene objectives. We also present a multiform ensemble enhancement (MEE) training method. Ensemble enhancement involves the concatenation of features extracted from different backbones trained by a combination of multiform self-supervised auxiliary tasks. MEE can not only be regarded as a more straightforward alternative to knowledge distillation but also can achieve an effective compromise between expensive computational cost and classification accuracy. In addition, we provide two scene classification schemes of inductive and transductive settings, corresponding to solving the difficulties of remote sensing data acquisition and labeling. The proposed network achieves state-of-the-art results on three benchmark FSRSSC datasets. The potential of the MES2L framework is also demonstrated in combination with classical metalearning-based and metric learning-based few-shot algorithms.

Super-resolution (SR) is one of the powerful techniques to improve image quality for low-resolution (LR) hyperspectral image (HSI) with insufficient detail and noise. Traditional methods typically perform simple cascade or addition during the fusion of the auxiliary high-resolution RGB and LR HSI. As a result, the abundant HR RGB details are not utilized as a priori information to enhance the HSI feature representation, leaving room for further improvements. To address this issue, we propose an RGB-induced feature modulation network for HSI SR (IFMSR). Considering that similar patterns are common in images, a multi-corresponding patch aggregation is designed to globally assemble this contextual information, which is beneficial for feature learning. Besides, to adequately exploit plentiful HR RGB details, an RGB-induced detail enhancement (RDE) module and a deep cross-modality feature modulation (CFM) module are proposed to transfer the supplementary materials from RGB to HSI. These modules can provide a more direct and instructive representation, leading to further edge recovery. Experiments on several datasets demonstrate that our approach achieves comparable performance under more realistic degradation condition. Our code is publicly available at https://github.com/qianngli/IFMSR.

Convolutional neural networks (CNNs) have been adopted to classify the remote sensing scene image. However, the application of these complicated networks on the satellite platform is difficult because of the limited computation resources. Therefore, we propose an energy-based filter pruning framework (EFPF) to reduce the size of the original model. The energy can be obtained through the eigenvalues of each weight tensor by singular value decomposition (SVD). Specifically, we calculate the energy of each layer by the ratio of eigenvalues that are lower than a specified truncation parameter and then remove filters from the original layer in light of the degree of energy. The EFPF is reliable because SVD techniques can capture the covariance among all filters from the original weight tensor, and therefore, the energy from the eigenvalues can reflect the redundancy of the filters. (i.e., if the distribution of eigenvalues is sharp, then the energy among filters will be lower, and the redundancy will be higher.) Surprisingly, the EFPF can reduce the FLOPs and parameters, as well as improve the top1 accuracy obviously when the VGG-16 and ResNet-50 are adopted to classify the AID, NWPU45, PatternNet, and WHU19 datasets. Additionally, the EFPF can achieve similar pruning results when the original model is fully-trained (converge) and under-trained (In-converge), which means we can save the training computation resources for the original model.

In recent years, deep learning models, which possess powerful feature extraction abilities, have achieved remarkable success in the classification of hyperspectral images (HSIs). Nevertheless, a common challenge faced by most deep learning models, including few-shot learning models, is the scarcity of valid labeled samples. To address this issue, we propose a cross-domain self-taught network (CDSTN) for few-shot hyperspectral image classification. The proposed CDSTN merges domain adaptation and semi-supervised self-taught strategy to implement the few-shot learning, which utilizes adequate labeled and unlabeled samples from source as well as target domain respectively. For the feature information extraction of HSI, we propose a deep spatial-spectral feature embedded extractor composed of four residual blocks and a channel attention module. Additionally, a set of domain classifiers are introduced behind each residual block for the purpose of domain alignment by extracting more domain information at different depths of the network. Finally, plenty of unlabeled samples are assigned with pseudo labels through the trained network, and a pseudo label refinement module is designed to select the most confident pseudo label sample for each class to further enrich the labeled database of target domain. Experiments conducted on four widely used benchmark HSI data sets demonstrate that CDSTN can obtain superior and stable performance with limited labeled samples compared with some state of the arts.

Recently, maximizing mutual information has emerged as a powerful tool for unsupervised graph representation learning. Existing methods are typically effective in capturing graph information from the topology view but consistently ignore the node feature view. To circumvent this problem, we propose a novel method by exploiting mutual information maximization across feature and topology views. Specifically, we first construct the feature graph to capture the underlying structure of nodes in feature spaces by measuring the distance between pairs of nodes. Then we use a cross-view representation learning module to capture both local and global information content across feature and topology views on graphs. To model the information shared by the feature and topology spaces, we develop a common representation learning module by using mutual information maximization and reconstruction loss minimization. Here, minimizing reconstruction loss forces the model to learn the shared information of feature and topology spaces. To explicitly encourage diversity between graph representations from the same view, we also introduce a disagreement regularization to enlarge the distance between representations from the same view. Experiments on synthetic and real-world datasets demonstrate the effectiveness of integrating feature and topology views. In particular, compared with the previous supervised methods, the proposed method achieves comparable or even better performance under the unsupervised representation and linear evaluation protocol.

Graph Neural Network (GNN) is capable of applying deep neural networks to graph domains. Recently, Message Passing Neural Networks (MPNNs) have been proposed to generalize several existing graph neural networks into a unified framework. For graph representation learning, MPNNs first generate discriminative node representations using the message passing function and then read from the node representation space to generate a graph representation using the readout function. In this paper, we analyze the representation capacity of the MPNNs for aggregating graph information and observe that the existing approaches ignore the self-loop for graph representation learning, leading to limited representation capacity. To alleviate this issue, we introduce a simple yet effective propagation enhanced extension, Self-Connected Neural Message Passing (SC-NMP), which aggregates the node representations of the current step and the graph representation of the previous step. To further improve the information flow, we also propose a Densely Self-Connected Neural Message Passing (DSC-NMP) that connects each layer to every other layer in a feed-forward fashion. Both proposed architectures are applied at each layer and the graph representation can then be used as input into all subsequent layers. Remarkably, combining these two architectures with existing GNN variants can improve these models’ performance for graph representation learning. Extensive experiments on various benchmark datasets strongly demonstrate the effectiveness, leading to superior performance for graph classification and regression tasks.

Graph matching endeavors to find corresponding nodes across two or more graphs, which plays a fundamental role in many vision and pattern matching tasks. However, existing graph matching algorithms often meet abnormal graphs with missing node features and suffer from numerous cluttered outliers in practical applications. To address these, we propose a novel deep graph matching method called Random Deep Graph Matching (RDGM). Different from the deterministic affinity inference in existing deep graph matching methods, RDGM performs message passing in a random manner during model training through randomly masking some available node features in the source or target graph, so that the affinity inference between nodes is insensitive to specific neighborhoods. In addition, a hierarchical attention graph neural network framework is devised in the node embedding process of RDGM, which can obtain more sufficient high-order structural information to reduce the impact of latent noise on affinity learning. Extensive experiments suggest that the proposed RDGM outperforms state-of-the-art graph matching methods, and demonstrates strong robustness and generalization performance.

Graph neural networks (GNNs) have demonstrated great success in many graph data-based applications. The impressive behavior of GNNs typically relies on the availability of a sufficient amount of labeled data for model training. However, in practice, obtaining a large number of annotations is prohibitively labor-intensive and even impossible. Co-training is a popular semi-supervised learning (SSL) paradigm, which trains multiple models based on a common training set while augmenting the limited amount of labeled data used for training each model via the pseudolabeled data generated from the prediction results of other models. Most of the existing co-training works do not control the quality of pseudolabeled data when using them. Therefore, the inaccurate pseudolabels generated by immature models in the early stage of the training process are likely to cause noticeable errors when they are used for augmenting the training data for other models. To address this issue, we propose a self-paced co-training for the GNN (SPC-GNN) framework for semi-supervised node classification. This framework trains multiple GNNs with the same or different structures on different representations of the same training data. Each GNN carries out SSL by using both the originally available labeled data and the augmented pseudolabeled data generated from other GNNs. To control the quality of pseudolabels, a self-paced label augmentation strategy is designed to make the pseudolabels generated at a higher confidence level to be utilized earlier during training such that the negative impact of inaccurate pseudolabels on training data augmentation, and accordingly, the subsequent training process can be mitigated. Finally, each of the trained GNN is evaluated on a validation set, and the best-performing one is chosen as the output. To improve the training effectiveness of the framework, we devise a pretraining followed by a two-step optimization scheme to train GNNs. Experimental results on the node classification task demonstrate that the proposed framework achieves significant improvement over the state-of-the-art SSL methods.

Audio and vision are two main modalities in video data. Multimodal learning, especially for audiovisual learning, has drawn considerable attention recently, which can boost the performance of various computer vision tasks. However, in video summarization, most existing approaches just exploit the visual information while neglecting the audio information. In this brief, we argue that the audio modality can assist vision modality to better understand the video content and structure and further benefit the summarization process. Motivated by this, we propose to jointly exploit the audio and visual information for the video summarization task and develop an audiovisual recurrent network (AVRN) to achieve this. Specifically, the proposed AVRN can be separated into three parts: 1) the two-stream long-short term memory (LSTM) is used to encode the audio and visual feature sequentially by capturing their temporal dependency; 2) the audiovisual fusion LSTM is used to fuse the two modalities by exploring the latent consistency between them; and 3) the self-attention video encoder is adopted to capture the global dependency in the video. Finally, the fused audiovisual information and the integrated temporal and global dependencies are jointly used to predict the video summary. Practically, the experimental results on the two benchmarks, i.e., SumMe and TVsum, have demonstrated the effectiveness of each part and the superiority of AVRN compared with those approaches just exploiting visual information for video summarization.

Recently, a number of resource allocation strategies have been proposed for evolutionary algorithms to efficiently tackle multiobjective optimization problems (MOPs). However, these methods mainly allocate computational resources based on the convergence improvement under the decomposition-based framework, which may become ineffective with the increased number of optimization objectives. To address this problem, this article suggests an immune-inspired resource allocation strategy, which breaks through the decomposition-based framework and can better balance convergence and diversity for many-objective optimization. In our method, the diversity distances of solutions are defined by the Euclidean distances of their projected points on the unit hyperplane. Then, based on the diversity distances, resource allocation is realized by using an immune cloning operator to encourage exploring sparse regions of the search space. Moreover, to provide high-quality solutions in coordination with this immune cloning operator, a novel archive update mechanism is designed. When compared to most well-known resource allocation strategies, our method is advantageous for many-objective optimization. The experimental results also validate the superiority of our method over several state-of-the-art evolutionary algorithms for solving two sets of complicated MOPs having 5 to 15 objectives.

The surrogate-assisted evolutionary algorithm (SAEA) is one of the most efficient approaches for solving expensive optimization problems. However, it still faces challenges when dealing with complex and high-dimensional problems. To fill this gap, a new algorithm (called SAMFEO) that combines surrogate-assisted and model-free evolutionary optimization is proposed in this article. SAMFEO consists of a local surrogate-assisted multioperator evolutionary optimization (LSA-MoEO) and a model-free single-operator evolutionary optimization (MF-SoEO). Specifically, LSA-MoEO adopts multiple evolutionary operators to generate a set of offspring and prescreens the best one as the final offspring by using a lightweight local surrogate model trained by some newest evaluated solutions. MF-SoEO follows the traditional evolutionary optimization paradigm and is triggered based on the optimization utility of the LSA-MoEO. It plays a crucial role in preventing the population from getting stagnation. Experimental results show that SAMFEO has significant advantages over several state-of-the-art SAEAs on some complex benchmark problems and one real-world problem.

In this paper, we propose a new change detection method of multi-temporal synthetic aperture radar (SAR) images. Due to the ability of extracting key feature of images and robustness to noise, stacked denoising auto encoder (SDAE) has been widely used in remote sensing. However, the single SDAE stills has some limitations to handle with the speckle noise of SAR images. Therefore, we propose a new structure Bi-hierarchical SDAE for feature extraction. The first level of SDAE denoises the original image and reconstructs the difference map, and the second level extracts the superpixel-based difference features for classification. Besides, Bayesian optimization effectively improves the classification performance of feature classifier. The experimental results of the datasets in this paper show that the Bi-hierarchical SDAE and Bayesian optimization framework has high accuracy and proves its effectiveness.

In recent years, convolutional neural networks (CNNs) have shown excellent effectiveness on hyperspectral image classification (HSI) tasks. However, it is a challenge to design a suitable CNN architecture to obtain great performance according to different tasks. Different from the traditional manual design, in this paper, an evolutionary multitasking CNN architecture search framework for HSI classification is proposed to search the optimal architectures and accomplish classification of different tasks simultaneously. Through encoding the CNN architectures, the proposed algorithm is able to achieve global search in the same search space and select well-adapted individuals for evolution. In the evolutionary multitasking environment, information can be transferred between and within tasks, which can accelerate the convergence and explore good architectures through beneficial transfer. In the experiments, the effectiveness of the proposed method is demonstrated by the comparison with different methods on two common data sets.

Abstract For the existing jamming discrimination methods on the multistatic radar system, the single feature of target echo space correlation is utilised as the metric, which leads to the lack of comprehensive feature extraction and universal discrimination algorithm. In this study, a discrimination method in a multistatic radar system based on the convolutional neural network is proposed. This proposal combines the advantages of multiple‐radar systems cooperative detection technology with the convolutional neural network, and effectively applies to the field of anti‐deception jamming, which takes full advantage of unknown information of echo data to obtain multi‐dimensional, comprehensive, complete and deep feature differences besides correlation, so as to achieve a better jamming discrimination effect. The simulation results show that the proposed method can extract the multidimensional and separable essential features of echoes, and all these features have a strong degree of differentiation between targets and jamming, which effectively reduce the influence of noise and pulse number. At the same time, the influence of radar distribution on jamming discrimination under non‐ideal conditions is relieved, when the correlation coefficient of the true target reaches 0.4, the discrimination probability remains above 85%, which broadens the boundary conditions of the application process.

Landslide inventory mapping (LIM) on the basis of change detection techniques has potential significance for landslide disaster analysis. In this letter, a novel LIM approach based on the adaptive histogram-mean distance (AHMD) is proposed, which adaptively considers spatial contextual information of different landslide regions to improve the detection performance. First, to adapt the shape, size, and distribution of various landslides, an adaptive region around a pixel is extracted by a novel adaptive region extension algorithm without parameter setting. Second, the pixels within the adaptive region are taken to construct the spectral frequency histograms, and then, the adaptive histogram mean (AHM) is developed as the feature of a histogram. Third, the AHMD is defined based on the bin-to-bin (B2B) distance to measure change magnitude between the pairwise AHMs. Finally, LIM can be obtained by a supervised threshold method called double-window flexible pace search (DFPS). Experimental results tested on two real datasets with a very high spatial resolution (VHR) demonstrate the outperformance of the proposed AHMD approach with seven comparative methods.

With the increasing popularity of deep learning, most recent developments of change detection (CD) approaches have taken advantage of deep learning techniques to improve the detection performance. However, it is usually necessary to elaborately design the network architecture and train the model with a large amount of labeled data, which are difficult to obtain in practice. To overcome these limitations, this letter proposed an unsupervised CD framework for high-resolution remote sensing images integrating transfer learning-based bilinear convolutional neural networks (BCNNs) and object-based change analysis. A difference image is first generated, which is used for the subsequent preclassification and superpixel segmentation. Then, two sets of superpixel samples with reliable labels derived from the bitemporal remote sensing images are input into two pretrained CNNs to extract representative features, respectively. On this basis, the matrix outer product is utilized to generate the combined bilinear features, which are input into the softmax classifier to discriminate the change and no-change information and thus obtaining the final change map by feeding all sample data into the well-trained model. The experimental results on three real data sets demonstrate the effectiveness and superiority of the proposed method over several existing CD approaches.

Cancer progression is dynamic, and tracking dynamic modules is promising for cancer diagnosis and therapy. Accumulated genomic data provide us an opportunity to investigate the underlying mechanisms of cancers. However, as far as we know, no algorithm has been designed for dynamic modules by integrating heterogeneous omics data. To address this issue, we propose an integrative framework for dynamic module detection based on regularized nonnegative matrix factorization method (DrNMF) by integrating the gene expression and protein interaction network. To remove the heterogeneity of genomic data, we divide the samples of expression profiles into groups to construct gene co-expression networks. To characterize the dynamics of modules, the temporal smoothness framework is adopted, in which the gene co-expression network at the previous stage and protein interaction network are incorporated into the objective function of DrNMF via regularization. The experimental results demonstrate that DrNMF is superior to state-of-the-art methods in terms of accuracy. For breast cancer data, the obtained dynamic modules are more enriched by the known pathways, and can be used to predict the stages of cancers and survival time of patients. The proposed model and algorithm provide an effective integrative analysis of heterogeneous genomic data for cancer progression.

Deep convolutional neural networks (CNNs) have achieved tremendous successes but tend to suffer from high computation costs mainly due to heavy over-parameterization, resulting in the difficulty of directly applying them to the ever-growing application demands based on low-end edge devices with strong power restriction and real-time inference requirement. Recently, there has much research attention devoted to compressing the network via pruning to address this issue. Most of the existing methods rely on some hand-designed pruning rules, which suffer from several limitations. Firstly, manually designed rules are only applicable to limited application scenarios, which can hardly generalize well in a broader scope. And these rules are typically designed based on human experience and via trial and error, and thus highly subjective. Then, channels of different layers in a network may have diverse distributions, which means the same pruning rule is not appropriate for each layer. To address these limitations, we propose a novel channel pruning scheme, in which the task-irrelevant channels are removed in a task-driven manner. Specifically, an adaptively differentiable search module is proposed to find the best pruning rule automatically for different layers in CNNs under sparsity constraints. Besides, we employed knowledge distillation to alleviate the excessive performance loss. Once the training process is finished, a compact network will be obtained by removing channels based on layer-wise pruning rules. We have evaluated the proposed method on some well-known benchmark datasets including CIFAR, MNIST, and ImageNet in comparison to several state-of-the-art pruning methods. Experimental results demonstrate the superiority of our method over the compared ones in terms of both parameters and FLOPs reduction.

Deep learning-based image inpainting has been widely studied, leading to great success. However, many methods adopt convolution and normalization operations, which will bring up some issues to affect the performance. The vanilla normalization cannot distinguish the pixels in corrupted regions from the other valid pixels, resulting in the mean and variance shifts. In addition, the limited receptive field of convolution makes it unable to capture long-range valid information directly. In order to tackle these challenges, we propose a novel deep generative model for image inpainting with two key modules, namely, the channel and spatially adaptive batch normalization (CSA-BN) module, and the selective latent-space-mapping-based contextual attention (SLSM-CA) layer. We replace the vanilla normalization with the CSA-BN module. By channel and spatially adaptive denormalization, the CSA-BN module can mitigate the spatial mean and variance shifts in each channel in a targeted way. In addition, we also integrate the SLSM-CA layer into our model to capture the long-range correlations explicitly. By introducing dual-branch attention and a feature selection module, the SLSM-CA layer can selectively utilize the multi-scale background information to improve prediction quality. What’s more, it introduces the latent spaces to achieve the low-rank approximations of attention matrices and to reduce computational costs. Extensive quantitative and qualitative evaluations demonstrate the superiority of the proposed method compared with state-of-the-art methods.

Detecting anomalies in videos is a fundamental issue in public security. The majority of existing deep learning methods often perform anomaly detection based on the behavior or the trajectory of a single target. However, due to the overlaps of the crowd and the low-resolution of monitoring images, the segmentation of population is hard to implement and the features cannot be learned thoroughly, which make the methods be easily disturbed by visual elements and thus may lead to false detection sometimes. To tackle these problems, we propose the influence-aware attention to learn the representative attributes of the whole crowd. Walking pedestrians can be divided into numbers of flows, and in this paper, we aim to measure the consistency of movement patterns in the same stream and the interactions between different streams. Meanwhile, great importance is given to the relation between pedestrians and the circumstance for certain anomalies occur as a result of environmental issues. Specifically, the influence-aware attention module is composed of the motion attention and the location attention, which is designed to quantify the relations in the scene from spatial and temporal aspects. For the lack of abnormal samples, we utilize a dual generator-based framework to learn interactions among normal scenes. Experimental results on six benchmarks verify the effectiveness and robustness of our proposed method.

Artificial neural networks inspired from the learning mechanism of the brain have achieved great successes in machine learning, especially those with deep layers. The commonly used neural networks follow the hierarchical multilayer architecture with no connections between nodes in the same layer. In this article, we propose a new group architectures for neural-network learning. In the new architecture, the neurons are assigned irregularly in a group and a neuron may connect to any neurons in the group. The connections are assigned automatically by optimizing a novel connecting structure learning probabilistic model which is established based on the principle that more relevant input and output nodes deserve a denser connection between them. In order to efficiently evolve the connections, we propose to directly model the architecture without involving weights and biases which significantly reduce the computational complexity of the objective function. The model is optimized via an improved particle swarm optimization algorithm. After the architecture is optimized, the connecting weights and biases are then determined and we find the architecture is robust to corruptions. From experiments, the proposed architecture significantly outperforms existing popular architectures on noise-corrupted images when trained only by pure images.

Performance of multi/many-objective evolutionary algorithms (MOEAs) based on decomposition is highly impacted by the Pareto front (PF) shapes of multi/many-objective optimization problems (MOPs), as their adopted weight vectors may not properly fit the PF shapes. To avoid this mismatch, some MOEAs treat solutions as weight vectors to guide the evolutionary search, which can adapt to the target MOP's PF automatically. However, their performance is still affected by the similarity metric used to select weight vectors. To address this issue, this article proposes a fuzzy decomposition-based MOEA. First, a fuzzy prediction is designed to estimate the population's shape, which helps to exactly reflect the similarities of solutions. Then, N least similar solutions are extracted as weight vectors to obtain N constrained fuzzy subproblems ( N is the population size), and accordingly, a shared weight vector is calculated for all subproblems to provide a stable search direction. Finally, the corner solution for each of m least similar subproblems ( m is the objective number) is preserved to maintain diversity, while one solution having the best aggregated value on the shared weight vector is selected for each of the remaining subproblems to speed up convergence. When compared to several competitive MOEAs in solving a variety of test MOPs, the proposed algorithm shows some advantages at fitting their different PF shapes.

The searching ability of the population-based search algorithms strongly relies on the coordinate system on which they are implemented. However, the widely used coordinate systems in the existing multifactorial optimization (MFO) algorithms are still fixed and might not be suitable for various function landscapes with differential modalities, rotations, and dimensions; thus, the intertask knowledge transfer might not be efficient. Therefore, this article proposes a novel intertask knowledge transfer strategy for MFOs implemented upon an active coordinate system that is established on a common subspace of two search spaces. The proper coordinate system might identify some common modality in a proper subspace to some extent. In this article, to seek the intermediate subspace, we innovatively introduce the geodesic flow that starts from a subspace, reaching another subspace in unit time. A low-dimension intermediate subspace is drawn from a uniform distribution defined on the geodesic flow, and the corresponding coordinate system is given. The intertask trial generation method is applied to the individuals by first projecting them on the low-dimension subspace, which reveals the important invariant features of the multiple function landscapes. Since intermediate subspace is generated from the major eigenvectors of tasks' spaces, this model turns out to be intrinsically regularized by neglecting the minor and small eigenvalues. Therefore, the transfer strategy can alleviate the influence of noise led by redundant dimensions. The proposed method exhibits promising performance in the experiments.

Recently, deep-learning-based feature extraction (FE) methods have shown great potential in hyperspectral image (HSI) processing. Unfortunately, it also brings a challenge that the training of the deep learning networks always requires large amounts of labeled samples, which is hardly available for HSI data. To address this issue, in this article, a novel unsupervised deep-learning-based FE method is proposed, which is trained in an end-to-end style. The proposed framework consists of an encoder subnetwork and a decoder subnetwork. The structure of the two subnetworks is symmetric for obtaining better downsampling and upsampling representation. Considering both spectral and spatial information, 3-D all convolution nets and deconvolution nets are used to structure the encoder subnetwork and decoder subnetwork, respectively. However, 3-D convolution and deconvolution kernels bring more parameters, which can deteriorate the quality of the obtained features. To alleviate this problem, a novel cost function with a sparse regular term is designed to obtain more robust feature representation. Experimental results on publicly available datasets indicate that the proposed method can obtain robust and effective features for subsequent classification tasks.

Surrogate-assisted evolutionary algorithms (SAEAs) have become very popular for tackling computationally expensive multiobjective optimization problems (EMOPs), as the surrogate models in SAEAs can approximate EMOPs well, thereby reducing the time cost of the optimization process. However, with the increased number of decision variables in EMOPs, the prediction accuracy of surrogate models will deteriorate, which inevitably worsens the performance of SAEAs. To deal with this issue, this article suggests an ensemble surrogate-based framework for tackling EMOPs. In this framework, a global surrogate model is trained under the entire search space to explore the global area, while a number of surrogate submodels are trained under different search subspaces to exploit the subarea, so as to enhance the prediction accuracy and reliability. Moreover, a new infill sampling criterion is designed based on a set of reference vectors to select promising samples for training the models. To validate the generality and effectiveness of our framework, three state-of-the-art evolutionary algorithms [nondominated sorting genetic algorithm III (NSGA-III), multiobjective evolutionary algorithm based on decomposition with differential evolution (MOEA/D-DE) and reference vector-guided evolutionary algorithm (RVEA)] are embedded, which significantly improve their performance for solving most of the test EMOPs adopted in this article. When compared to some competitive SAEAs for solving EMOPs with up to 30 decision variables, the experimental results also validate the advantages of our approach in most cases.

Hyperspectral images (HSIs) contain rich spectral signatures that reveal more image details and, thus, enable the detection of less noticeable changes on the ground. However, HSI-based change detection (CD) is susceptible to a large amount of irrelevant or noisy spectral and spatial information due to massive spectral bands. To address these issues, we propose a novel spectral and spatial attention network (S2AN) for HSI-based CD, which is capable to suppress CD-irrelevant spectral and spatial information via adaptive spectral and spatial attention mechanisms. S2AN takes as input the image patch from the difference map between two HSIs and outputs the status of change for the patch. Specifically, S2AN is composed of several repeated attention blocks, each of which contains the spectral attention (SpeA) module for directly calculating the attention score for each input channel, the Gaussian spatial attention (GSpaA) module that first constructs an adaptive Gaussian distribution and then samples it to derive the attention scores for each spatial position, and the convolutional feature extraction (CFE) module for extracting features from the attention-weighted input. It is worth mentioning that, in addition to the advantage of the attention, GSpaA also reduces the sensitivity of patch size for patch-based methods. To effectively train S2AN when facing insufficient labeled data, a semisupervised strategy that combines supervised and unsupervised methods to augment labeled training data is proposed. Experiments on several HSI datasets in comparison to existing methods show the superiority of S2AN.

The few-shot scene classification is dedicated to identifying unseen remote sensing classes when only a very small number of labeled samples are available for reference. Most of the existing few-shot scene classification methods are based on meta-learning and employ the episodic learning for training, which lacks the consideration for the utilization of data efficiency. In this paper, instead of designing sophisticated meta-learning based algorithms, we are committed to training a feature extractor with good generalization performance and strong feature extraction capability. Specifically, we propose a novel two-path aggregation attention network with quad-patch data augmentation, called DANet, to solve the problem of few-shot scene classification from both data and architecture aspects. In terms of data, we design a new data augmentation strategy named quad-patch augmentation. We utilize the characteristics of remote sensing images to chunk and reassemble any existing data, thereby generating pseudo-new data to enrich the training set. In terms of architecture, we present a two-path aggregation attention module that makes it easier for the model to focus on the key clues in a targeted manner. The comparative experiments in natural image datasets and remote sensing image datasets demonstrate the effectiveness of our two innovations. In addition, DANet achieves competitive or state-of-the-art (SOTA) results on three benchmark scene classification datasets.

Hyperspectral unmixing including endmember extration and abundance estimation has been investigated successively in recent years due to increasingly hyperspectral processing requirements. As one type of decision after solution paradigm, multiobjective endmember estimation method is able to obtain a set of Pareto optimal solutions, thus providing a wealth of information to determine the most representative endmembers. In addition, multiobjective optimization methods also have the characteristics of flexible modeling, excellent global convergence and commendable adaptability, etc. However, the evolutionary algorithms designed for this kind of method generally use little spatial-spectral information of the hyperspectral image. In this paper, we delve into the memetic strategy by exploiting the topological structure of hyperspectral data in the high-dimensional space to establish a vertex-directed multiobjective endmember estimation method, termed VD-MoEE. According to the frequently used linear mixture model, endmembers of hyperspectral images are generally distributed at the vertices of hyperspectral data manifold. Therefore, we design a vertex-directed local search operator to guide the search direction of the individuals in evolutionary algorithms. Experimental results on synthetic as well as real data sets demonstrated that the proposed VD-MOEE is able to achieve appealing performance in terms of the solution selecting and accuracy in comparison with several classic and state-of-the-art endmember estimation methods.

Single hyperspectral image (HSI) super-resolution (SR) methods using a auxiliary high-resolution (HR) RGB image have achieved great progress recently. However, most existing methods aggregate the information of RGB image and HSI early during input or shallow feature extraction, whose difference between two images has not been treated and discussed. Although a few methods combine both the image features in the middle layer of the network, they fail to make full use of the two inherent properties, i.e., rich spectra of HSI and HR content of RGB image, to guide model representation learning. To address these issues, in this article, we propose a dual-stage learning approach for HSI SR to learn a general spatial–spectral prior and image-specific details, respectively. In the coarse stage, we fully take advantage of two adjacent bands and RGB image to build the model. During coarse SR, a symmetrical feature propagation approach is developed to learn the inherent content of each image over a relatively long range. The symmetrical structure encourages the two streams to better retain their particularity. Meanwhile, it can realize the information interaction by the adaptive local block aggregation (ALBA) module. To learn image-specific details, a back-projection refinement network is embedded in the structure, which further improves the performance in fine stage. The experiments on four benchmark datasets demonstrate that the proposed approach presents excellent performance over the existing methods. Our code is publicly available athttps://github.com/qianngli/SFPN.

Building change detection (BCD) for very-high-spatial-resolution (VHR) remote sensing images is very important and challenging in the field of remote sensing, as the building is one of the most significant and valuable man-made ground targets. This article proposes a local–global pyramid network (LGPNet) that combines a local feature pyramid module (LFPM) and a global spatial pyramid module (GSPM) for various building feature extraction. The LFPM is constructed using the convolutional kernel with three different pyramid scales, and then, the local pyramid features are obtained by adding features of each scale. In the GSPM, the global spatial pyramid features are extracted by adaptive average pooling to acquire global contextual information from different fields of view on deep features. The LFPM and the GSPM work in a parallel and complementary manner to capture discriminative features of various buildings. In addition to the LFPM and the GSPM, the proposed LGPNet also employs two general attention mechanisms, i.e., the position attention module and the channel attention module, which can select and emphasize adaptively some building features with high semantic responses. Besides, in order to mitigate the influence of other ground targets to a certain extent, a cross-task transfer learning strategy is introduced to make the LGPNet focus on the building, which significantly improves the performance of our method. Extensive experiments on two public available BCD datasets show that the proposed LGPNet can achieve significant improvement compared with eight other state-of-the-art methods. The source code and the pretrained model will be released athttps://github.com/TongfeiLiu/LGPNet.

Extracting 3D information from a single optical image is very attractive. Recently emerging self-supervised methods can learn depth representations without using ground truth depth maps as training data by transforming the depth prediction task into an image synthesis task. However, existing methods rely on a differentiable bilinear sampler for image synthesis, which results in each pixel in a synthetic image being derived from only four pixels in the source image and causes each pixel in the depth map to perceive only a few pixels in the source image. In addition, when calculating the photometric error between a synthetic image and its corresponding target image, existing methods only consider the photometric error within a small neighborhood of each single pixel and therefore ignore correlations between larger areas, which causes the model to tend to fall into the local optima for small patches. In order to extend the perceptual area of the depth map over the source image, we propose a novel multi-scale method that downsamples the predicted depth map and performs image synthesis at different resolutions, which enables each pixel in the depth map to perceive more pixels in the source image and improves the performance of the model. As for the locality of photometric error, we propose a structural similarity (SSIM) pyramid loss to allow the model to sense the difference between images in multiple areas of different sizes. Experimental results show that our method achieves superior performance on both outdoor and indoor benchmarks.

Representing nodes in a network as low-dimensional dense vectors can facilitate the analysis of complex networks, which is a challenging task and has attracted increasing attention. However, in the real world, networks are changing over time, such as cooperation in citation networks and communication in email networks. Most of the recent embedding methods only focus on static networks. Thus they ignore the critical temporal information, which serves as a supplement to structure information and has been proved to improve the quality of node embedding. In this work, we propose an unsupervised deep learning model called DTINE, which explores temporal information for further enhancing the robustness of node representations in dynamic networks. To preserve network topology, we pertinently design a temporal weight and sampling strategy to extract features from the neighborhoods. An attention mechanism will be applied on the recurrent neural network to measure the contributions of historical information and capture the evolution of the networks. Experimental results on four real-world networks demonstrate that the proposed method achieves better performance than state-of-the-art methods.

Network embedding has been widely used to solve the network analytics problem. Existing methods mainly focus on networks with single-layered homogeneous or heterogeneous networks. However, many real-world complex systems can be naturally represented by multilayer networks, which is another term of heterogeneous networks with multiple edge/relation types. The problem of how to capture and utilize rich interaction information of multi-type relations causes a major challenge of multilayer network embedding. To address this problem, we propose a fast and scalable multilayer network embedding model, called HMNE, to efficiently preserve and learn information of multi-type relations into a unified embedding space. We develop a heuristic 3D interactive walk technique dedicated for multilayer networks, which can leverage rich interactions among distinct layers and effectively capture important information contained in the layered structure. We evaluate our proposed model HMNE on two downstream analytic applications: node classification and link prediction. Experimental results on seven social and biological multilayer network datasets demonstrate that the proposed model outperforms existing competitive baselines with reduced time and memory occupations.

As a unified framework for graph neural networks, message passing-based neural network (MPNN) has attracted a lot of research interest and has been shown successfully in a number of domains in recent years. However, because of over-smoothing and vanishing gradients, deep MPNNs are still difficult to train. To alleviate these issues, we first introduce a deep hierarchical layer aggregation (DHLA) strategy, which utilizes a block-based layer aggregation to aggregate representations from different layers and transfers the output of the previous block to the subsequent block, so that deeper MPNNs can be easily trained. Additionally, to stabilize the training process, we also develop a novel normalization strategy, neighbor normalization (NeighborNorm), which normalizes the neighbor of each node to further address the training issue in deep MPNNs. Our analysis reveals that NeighborNorm can smooth the gradient of the loss function, i.e., adding NeighborNorm makes the optimization landscape much easier to navigate. Experimental results on two typical graph pattern-recognition tasks, including node classification and graph classification, demonstrate the necessity and effectiveness of the proposed strategies for graph message-passing neural networks.

Face is one of the most attractive sensitive information in visual shared data. It is an urgent task to design an effective face deidentification method to achieve a balance between facial privacy protection and data utilities when sharing data. Most of the previous methods for face deidentification rely on attribute supervision to preserve a certain kind of identity-independent utility but lose the other identity-independent data utilities. In this article, we mainly propose a novel disentangled representation learning architecture for multiple attributes preserving face deidentification called replacing and restoring variational autoencoders (R2VAEs). The R2VAEs disentangle the identity-related factors and the identity-independent factors so that the identity-related information can be obfuscated, while they do not change the identity-independent attribute information. Moreover, to improve the details of the facial region and make the deidentified face blends into the image scene seamlessly, the image inpainting network is employed to fill in the original facial region by using the deidentified face asa priori. Experimental results demonstrate that the proposed method effectively deidentifies face while maximizing the preservation of the identity-independent information, which ensures the semantic integrity and visual quality of shared images.

Network representation learning or embedding aims to project the network into a low-dimensional space that can be devoted to different network tasks. Temporal networks are an important type of network whose topological structure changes over time. Compared with methods on static networks, temporal network embedding (TNE) methods are facing three challenges: 1) it cannot describe the temporal dependence across network snapshots; 2) the node embedding in the latent space fails to indicate changes in the network topology; and 3) it cannot avoid a lot of redundant computation via parameter inheritance on a series of snapshots. To overcome these problems, we propose a novel TNE method named temporal network embedding method based on the VAE framework (TVAE), which is based on a variational autoencoder (VAE) to capture the evolution of temporal networks for link prediction. It not only generates low-dimensional embedding vectors for nodes but also preserves the dynamic nonlinear features of temporal networks. Through the combination of a self-attention mechanism and recurrent neural networks, TVAE can update node representations and keep the temporal dependence of vectors over time. We utilize parameter inheritance to keep the new embedding close to the previous one, rather than explicitly using regularization, and thus, it is effective for large-scale networks. We evaluate our model and several baselines on synthetic data sets and real-world networks. The experimental results demonstrate that TVAE has superior performance and lower time cost compared with the baselines.

Recently, stochastic hard thresholding (HT) optimization methods [e.g., stochastic variance reduced gradient hard thresholding (SVRGHT)] are becoming more attractive for solving large-scale sparsity/rank-constrained problems. However, they have much higher HT oracle complexities, especially for high-dimensional data or large-scale matrices. To address this issue and inspired by the well-known Gradient Support Pursuit (GraSP) method, this article proposes a new Relaxed Gradient Support Pursuit (RGraSP) framework. Unlike GraSP, RGraSP only requires to yield an approximation solution at each iteration. Based on the property of RGraSP, we also present an efficient stochastic variance reduction-gradient support pursuit algorithm and its fast version (called stochastic variance reduced gradient support pursuit (SVRGSP+). We prove that the gradient oracle complexity of both our algorithms is two times less than that of SVRGHT. In particular, their HT complexity is about$\kappa _{\widehat {s}}$times less than that of SVRGHT, where$\kappa _{\widehat {s}}$is the restricted condition number. Moreover, we prove that our algorithms enjoy fast linear convergence to an approximately global optimum, and also present an asynchronous parallel variant to deal with very high-dimensional and sparse data. Experimental results on both synthetic and real-world datasets show that our algorithms yield superior results than the state-of-the-art gradient HT methods.

Change detection based on heterogeneous images, such as optical images and synthetic aperture radar images, is a challenging problem because of their huge appearance differences. To combat this problem, we propose an unsupervised change detection method that contains only a convolutional autoencoder (CAE) for feature extraction and the commonality autoencoder for commonalities exploration. The CAE can eliminate a large part of redundancies in two heterogeneous images and obtain more consistent feature representations. The proposed commonality autoencoder has the ability to discover common features of ground objects between two heterogeneous images by transforming one heterogeneous image representation into another. The unchanged regions with the same ground objects share much more common features than the changed regions. Therefore, the number of common features can indicate changed regions and unchanged regions, and then a difference map can be calculated. At last, the change detection result is generated by applying a segmentation algorithm to the difference map. In our method, the network parameters of the commonality autoencoder are learned by the relevance of unchanged regions instead of the labels. Our experimental results on five real data sets demonstrate the promising performance of the proposed framework compared with several existing approaches.

Many real systems are represented in form of multiplex networks composed of a set of nodes, multiple layers of links, and coupling node relationships across all layers. These systems are very vulnerable to damages during both attacks and recoveries due to potential node cascading failures (NCFs). Although some progress has recently been made in studying network robustness and resilience, the comprehensive impacts of coupling node relationships and community structures on NCFs remain unclear. Accordingly, in this article, we study the robustness and resilience of multiplex networks in the presence of NCFs caused by coupling node relationships and community structures. We first model the failure processes of multiplex networks during both attacks and recoveries as node-community cascading failures (called NCCFs), and then theoretically demonstrate the fragility of multiplex networks to random node damages under NCCFs. Subsequently, to improve network robustness and resilience, we adopt a node protection strategy and propose a cost-aware constrained optimization problem. Finally, we devise a degree-based simulated annealing algorithm for solving this optimization problem. Extensive experiments on both simulated and real multiplex networks show that NCCFs make networks more vulnerable to unpredictable damage than classical NCFs. The results also show the superiority of the proposed algorithm over the state-of-the-art algorithms in improving network robustness and resilience.

The bottleneck of computation burden limits the widespread use of the 2nd order optimization algorithms for training deep neural networks. In this paper, we present a computationally efficient approximation for natural gradient descent, named Swift Kronecker-Factored Approximate Curvature (SKFAC), which combines Kronecker factorization and a fast low-rank matrix inversion technique. Our research aims at both fully connected and convolutional layers. For the fully connected layers, by utilizing the low-rank property of Kronecker factors of Fisher information matrix, our method only requires inverting a small matrix to approximate the curvature with desirable accuracy. For convolutional layers, we propose a way with two strategies to save computational efforts without affecting the empirical performance by reducing across the spatial dimension or receptive fields of feature maps. Specifically, we propose two effective dimension reduction methods for this purpose: Spatial Subsampling and Reduce Sum. Experimental results of training several deep neural networks on Cifar-10 and ImageNet-1k datasets demonstrate that SKFAC can capture the main curvature and yield comparative performance to K-FAC. The proposed method bridges the wall-clock time gap between the 1st and 2nd order algorithms.

Unsupervised domain adaptation aims to learn an accurate classifier for a target domain by leveraging knowledge learned from a related (source) domain. Existing approaches focus on deriving new domain-invariant feature representations to align two domains and an extra classifier is required. In this paper, we propose a novel unsupervised domain adaptation method to train a classifier directly for the target domain without learning the domain-invariant feature representation. For our method, the pseudo labels are assigned to target samples. An effective method is proposed to measure the relationship among cross-domain samples more accurately, so that we can construct a$p$-nearest neighbor graph. Then label propagation is employed to update the target sample labels. The graph model and labels of target samples are expected to be updated alternately within an iterative framework. To further improve the classifier, a fuzzy classification and pseudo-label selection mechanism are utilized. Extensive experiments validate that our proposed method is superior or comparable to the state-of-the-art unsupervised domain adaptation methods.

BACKGROUND: Gene prioritization (gene ranking) aims to obtain the centrality of genes, which is critical for cancer diagnosis and therapy since keys genes correspond to the biomarkers or targets of drugs. Great efforts have been devoted to the gene ranking problem by exploring the similarity between candidate and known disease-causing genes. However, when the number of disease-causing genes is limited, they are not applicable largely due to the low accuracy. Actually, the number of disease-causing genes for cancers, particularly for these rare cancers, are really limited. Therefore, there is a critical needed to design effective and efficient algorithms for gene ranking with limited prior disease-causing genes. RESULTS: In this study, we propose a transfer learning based algorithm for gene prioritization (called TLGP) in the cancer (target domain) without disease-causing genes by transferring knowledge from other cancers (source domain). The underlying assumption is that knowledge shared by similar cancers improves the accuracy of gene prioritization. Specifically, TLGP first quantifies the similarity between the target and source domain by calculating the affinity matrix for genes. Then, TLGP automatically learns a fusion network for the target cancer by fusing affinity matrix, pathogenic genes and genomic data of source cancers. Finally, genes in the target cancer are prioritized. The experimental results indicate that the learnt fusion network is more reliable than gene co-expression network, implying that transferring knowledge from other cancers improves the accuracy of network construction. Moreover, TLGP outperforms state-of-the-art approaches in terms of accuracy, improving at least 5%. CONCLUSION: The proposed model and method provide an effective and efficient strategy for gene ranking by integrating genomic data from various cancers.

Abstract Fine‐grained visual classification is challenging due to the inherently subtle intra‐class object variations. To solve this issue, a novel framework named channel attention and progressive multi‐granularity training network, is proposed. It first exploits meaningful feature maps through the channel attention module and captures multi‐granularity features by the progressive multi‐granularity training module. For each feature map, the channel attention module is proposed to explore channel‐wise correlation. This allows the model to re‐weight the channels of the feature map according to the impact of their semantic information on performance. Furthermore, the progressive multi‐granularity training module is introduced to fuse features cross multi‐granularity. And the fused features pay more attention to the subtle differences between images. The model can be trained efficiently in an end‐to‐end manner without bounding box or part annotations. Finally, comprehensive experiments are conducted to show that the method achieves state‐of‐the‐art performances on the CUB‐200‐2011, Stanford Cars, and FGVC‐Aircraft datasets. Ablation studies demonstrate the effectiveness of each part in our module.

The global alignment of biological networks (GABN) aims to find an optimal alignment between proteins across species, such that both the biological structures and the topological structures of the proteins are maximally conserved. The research on GABN has attracted great attention due to its applications on species evolution, orthology detection and genetic analyses. Most of the existing methods for GABN are difficult to obtain a good tradeoff between the conservation of the biological structures and topological structures. In this paper, we propose a multi-neighborhood learning method for solving GABN (called as CLMNA). CLMNA first models GABN as an optimization of a weighted similarity which evaluates the conserved biological and topological similarities of an alignment, and then it combines a first-proximity, second-proximity and individual-aware proximity learning algorithm to solve the modeled problem. Finally, systematic experiments on 10 pairs of biological networks across 5 species show the superiority of CLMNA over the state-of-the-art network alignment algorithms. They also validate the effectiveness of CLMNA as a refinement method on improving the performance of the compared algorithms.

Inspired by the learning rules of humans/animals, self-paced learning (SPL) ranks the samples from easy to complex and assigns real-valued weights to the samples. The current SPL regimes adopt an increasing pace parameter to select the training samples. Obviously, it is difficult to tune the pace parameter during iterations. Furthermore, the current SPL regimes cannot go back to the previous stage even when the current model is worse than the previous one. In this article, a novel Pareto SPL (PSPL) approach is proposed to address the above issues. In PSPL, the SPL problem is considered as a multiobjective optimization problem. Then, a Pareto-based differential evolution algorithm is utilized to optimize the self-paced function and the weighted loss function simultaneously. In PSPL, a new representation method is designed to assign nonpositive weights to the unselected samples. Then, a modified mutation operator inspired by the SPL idea is proposed to generate new population based on ranking the individuals and assigning weights. No pace parameters are introduced in the proposed technique and PSPL can obtain entire solution spectrum to automatically adjust the solution path. The effectiveness of the PSPL has been demonstrated through rigorous studies with matrix factorization and multiclass classification problems.

Retrieving nearest neighbors across correlated data in multiple modalities, such as image-text pairs on Facebook and video-tag pairs on YouTube, has become a challenging task due to the huge amount of data. Multimodal hashing methods that embed data into binary codes can boost the retrieving speed and reduce storage requirements. Since unsupervised multimodal hashing methods are usually inferior to supervised ones and the supervised ones require too much manually labeled data, the proposed method in this paper utilizes partially available labels to design a semisupervised multimodal hashing method. The labels for unlabeled data are treated as an interval type-2 fuzzy set and estimated by the available labels. By defuzzifying the estimated labels using hard partitioning, a supervised multimodal hashing method is used to generate binary codes. Experiments show that the proposed semisupervised method with 50% labels can get a medium performance among the compared supervised ones and achieve approximate performance to the best supervised method with 90% labels. With only 10% labels, the proposed method can still compete with the worst compared supervised one. Furthermore, the proposed label estimation method has been experimentally proven to be more feasible for a multilabeled MIRFlickr data set in a hash lookup task.

This article suggests a multimodal multiobjective evolutionary algorithm with dual clustering in decision and objective spaces. One clustering is run in decision space to gather nearby solutions, which will classify solutions into multiple local clusters. Nondominated solutions within each local cluster are first selected to maintain local Pareto sets, and the remaining ones with good convergence in objective space are also selected, which will form a temporary population with more than${N}$solutions (${N}$is the population size). After that, a second clustering is run in objective space for this temporary population to get${N}$final clusters with good diversity in objective space. Finally, a pruning process is repeatedly run on the above clusters until each cluster has only one solution, which removes the most crowded solution in decision space from the most crowded cluster in objective space each time. This way, the clustering in decision space can distinguish all Pareto sets and avoid the loss of local Pareto sets, while that in objective space can maintain diversity in objective space. When solving all the benchmark problems from the competition of multimodal multiobjective optimization in the IEEE Congress on Evolutionary Computation 2019, the experiments validate our advantages to maintain diversity in both objective and decision spaces.

This article proposes a novel and computationally efficient explicit intertask information transfer strategy between optimization tasks by aligning the subspaces. In evolutionary multitasking, the tasks might have biases embedded in function landscapes and decision spaces, which often causes the threat of predominantly negative transfer. However, the complementary information among different tasks can give an enhanced performance of solving complicated problems when properly harnessed. In this article, we distill this insight by introducing an intertask knowledge transfer strategy implemented in the low-dimension subspaces via a learnable alignment matrix. Specifically, to unveil the significant features of the function landscapes, the task-specific low-dimension subspaces is established based on the distribution information of subpopulations possessed by tasks, respectively. Next, the alignment matrix between pairwise subspaces is learned by minimizing the discrepancies of the subspaces. Given the aligned subspaces by applying the alignment matrix to subspaces' base vectors, the individuals from different tasks are then projected into aligned subspaces and reproduce therein. Moreover, since this method only considers the leading eigenvectors, it turns out to be intrinsically regularized and noise-insensitive. Comprehensive experiments are conducted on the synthetic and practical benchmark problems so as to assess the efficacy of the proposed method. According to the experimental results, the proposed method exhibits a superior performance compared with existing evolutionary multitask optimization algorithms.

Automated construction of deep neural networks (DNNs) has become a research hot spot nowadays because DNN’s performance is heavily influenced by its architecture and parameters, which are highly task-dependent, but it is notoriously difficult to find the most appropriate DNN in terms of architecture and parameters to best solve a given task. In this work, we provide an insight into the automated DNN construction process by formulating it into a multilevel multiobjective large-scale optimization problem with constraints, where the nonconvex, nondifferentiable, and black-box nature of this problem make evolutionary algorithms (EAs) to stand out as a promising solver. Then, we give a systematical review of existing evolutionary DNN construction techniques from different aspects of this optimization problem and analyze the pros and cons of using EA-based methods in each aspect. This work aims to help DNN researchers to better understand why, where, and how to utilize EAs for automated DNN construction and meanwhile, help EA researchers to better understand the task of automated DNN construction so that they may focus more on EA-favored optimization scenarios to devise more effective techniques.

In this paper, we study efficient differentially private alternating direction methods of multipliers (ADMM) via gradient perturbation for many centralized machine learning problems. For smooth convex loss functions with (non)-smooth regularization, we propose the first differentially private ADMM (DP-ADMM) algorithm with the performance guarantee of (ϵ,δ)-differential privacy ((ϵ,δ)-DP). From the viewpoint of theoretical analysis, we use the Gaussian mechanism and the conversion relationship between Rényi Differential Privacy (RDP) and DP to perform a comprehensive privacy analysis for our algorithm. Then we establish a new criterion to prove the convergence of the proposed algorithms including DP-ADMM. We also give the utility analysis of our DP-ADMM. Moreover, we propose a new accelerated DP-ADMM (DP-AccADMM) algorithm with the Nesterov’s acceleration technique. Finally, we conduct numerical experiments on many real-world datasets to show the privacy-utility tradeoff of the two proposed algorithms, and all the comparative analysis shows that DP-AccADMM converges faster and has a better utility than DP-ADMM, when the privacy budget ϵ is larger than a threshold.

Many complex systems derived from nature and society consist of multiple types of entities and heterogeneous interactions, which can be effectively modeled as heterogeneous information network (HIN). Structural analysis of heterogeneous networks is of great significance by leveraging the rich semantic information of objects and links in the heterogeneous networks. And, clustering heterogeneous networks aims to group vertices into classes, which sheds light on revealing the structure–function relations of the underlying systems. The current algorithms independently perform the feature extraction and clustering, which are criticized for not fully characterizing the structure of clusters. In this study, we propose a learning model by joint Graph Embedding and Nonnegative Matrix Factorization (aka GEjNMF ), where feature extraction and clustering are simultaneously learned by exploiting the graph embedding and latent structure of networks. We formulate the objective function of GEjNMF and transform the heterogeneous network clustering problem into a constrained optimization problem, which is effectively solved by l 0 -norm optimization. The advantage of GEjNMF is that features are selected under the guidance of clustering, which improves the performance and saves the running time of algorithms at the same time. The experimental results on three benchmark heterogeneous networks demonstrate that GEjNMF achieves the best performance with the least running time compared with the best state-of-the-art methods. Furthermore, the proposed algorithm is robust across heterogeneous networks from various fields. The proposed model and method provide an effective alternative for heterogeneous network clustering.

With the prevalence of the Internet, online reviews have become a valuable information resource for people. However, the authenticity of online reviews remains a concern, and deceptive reviews have become one of the most urgent network security problems to be solved. Review spams will mislead users into making suboptimal choices and inflict their trust in online reviews. Most existing research manually extracted features and labeled training samples, which are usually complicated and time-consuming. This paper focuses primarily on a neglected emerging domain - movie review, and develops a novel unsupervised spam detection model with an attention mechanism. By extracting the statistical features of reviews, it is revealed that users will express their sentiments on different aspects of movies in reviews. An attention mechanism is introduced in the review embedding, and the conditional generative adversarial network is exploited to learn users’ review style for different genres of movies. The proposed model is evaluated on movie reviews crawled from Douban, a Chinese online community where people could express their feelings about movies. The experimental results demonstrate the superior performance of the proposed approach.

Deep neural networks (DNNs), characterized by sophisticated architectures capable of learning a hierarchy of feature representations, have achieved remarkable successes in various applications. Learning DNN's parameters is a crucial but challenging task that is commonly resolved by using gradient-based backpropagation (BP) methods. However, BP-based methods suffer from severe initialization sensitivity and proneness to getting trapped into inferior local optima. To address these issues, we propose a DNN learning framework that hybridizes CC-based optimization with BP-based gradient descent, called BPCC, and implement it by devising a computationally efficient CC-based optimization technique dedicated to DNN parameter learning. In BPCC, BP will intermittently execute for multiple training epochs. Whenever the execution of BP in a training epoch cannot sufficiently decrease the training objective function value, CC will kick in to execute by using the parameter values derived by BP as the starting point. The best parameter values obtained by CC will act as the starting point of BP in its next training epoch. In CC-based optimization, the overall parameter learning task is decomposed into many subtasks of learning a small portion of parameters. These subtasks are individually addressed in a cooperative manner. In this article, we treat neurons as basic decomposition units. Furthermore, to reduce the computational cost, we devise a maturity-based subtask selection strategy to selectively solve some subtasks of higher priority. Experimental results demonstrate the superiority of the proposed method over common-practice DNN parameter learning techniques.

Endmember selection of hyperspectral images is a practical yet difficult task due to the high spectral resolution and low spatial resolution of the hyperspectral cameras. The paradigm of multitask optimization has been investigated over two decades, which aim to handle multiple tasks simultaneously. To address these issues, we propose a novel multitasking framework based on multiobjective optimization evolutionary algorithm based on decomposition (MOEA/D). Specifically, we use a single population to simultaneously perform multiple subset selection tasks and apply it to a specific scene-the endmember selection of hyperspectral images. It is natural to consider that pixels in a homogeneous region of hyperspectral image as a task. Then, a within-task and between-task genetic transfer operator is constructed to reinforce the exchange of genetic material belonging to the same or different tasks for better and quicker search of the decision space. After that, this algorithm obtains a set of nondominated solutions for better decision of the active endmembers. Experiments on hyperspectral datasets show the effectiveness of our method in finding the real active endmembers.

In this paper, inspired from the nuclei in brain, we propose a nucleus neural network (NNN) and corresponding connecting architecture learning method. In a nucleus, the neurons are not assigned as regular layers, i.e., a neuron may connect to any neurons in the nucleus and the connections are self-organized according to data distribution. This type of architecture gets rid of layer limitation and makes full use of processing capability of each neuron. It is crucial to assign connections between all the neuron pairs. To address the time demanding of objectives in traditional architecture learning methods, we propose an efficient architecture learning model for the nucleus based on the principle that more relevant input and output neuron pair deserves higher connecting density. The new objective measures the information flow through the network architecture without involvement of weights and biases which greatly reduces the computational complexity. We find that this novel architecture is robust to irrelevant components in test data. So we reconstruct a new dataset based on the MNIST dataset where the types of digital backgrounds in training and test sets are different. The new dataset is a great challenge for learners because training data and test data are not only independent but also follow different distributions. Experiments demonstrate that NNN achieves significant improvement over architectures with regular layers on the reconstructed dataset.

Graph classification is a practical problem in many different domains including bioinformatics, chemoinformatics, social network analysis, and etc. For the graph classification task, the existing graph neural network approaches usually generate graph features using graph pooling at each step. However, this strategy of pooling only at the current step ignores the impact of self-loop. To eliminate this limitation, we propose a novel self-loop graph pooling strategy that can utilize the node information of the current step and the graph representation information of the previous step to generate an effective representation for the graph classification task. Further to measure the importance of self-loop, we also develop a gated approach, gated graph pooling with self-loop, that utilizes the simple fusion gate to enhance the representation capacity of the model. We evaluate our model on common benchmark datasets and experimental results have demonstrated the superior performance improvement on predictive accuracy.

Domain adaptation aims at learning from the la-beled source domain to build an accurate classifier for a related but different target domain. Existing methods attempt to reduce domain discrepancy explicitly by means of statistical properties yet ignore the inherent differences among samples. In this paper, we present a novel solution for domain adaptation based on collaborative representation, named Discrepancy-Aware Collaborative Representation (DACR). Inspired by the success of nearest regularization, DACR develops a novel indicator to measure the discrepancy among every source sample and target domain. Then the indicator is employed in sparse regularization thus ensure that samples with small discrepancy have larger weights in the learned representation. Extensive experiments verify that DACR is able to achieve comparable performance with existing methods while significantly reducing computing complexity.

Effectively highlighting multiple changes in the earth surface from multi-temporal remote sensing images is a meaningful but challenging task. In order to reduce costs and ensure the performance, it is advisable to employ a semisupervised strategy to achieve this goal. As a discriminative joint classification task, semisupervised change detection aims to extract useful and discriminative features from a large amount of unlabeled data in addition to limited labeled samples. The discriminator of a well-trained generative adversarial network (GAN) is just right for this. Therefore, in this letter, we proposed a semisupervised GAN-based multiple change detection framework for multi-spectral images. First, the GAN is trained by all data without any prior information. Then, we combine two identical trained discriminators to construct a dual-pipeline joint classifier. Finally, the classifier is fine-tuned by a very small amount of labeled data to detect multiple changes. The superior performance of the proposed model over both real multi-spectral data sets demonstrates its robustness and effectiveness.

Accurate road centerline extraction is very important for many vital applications. In the road extraction, the acquisition of labeled data is time-consuming; thus, there is only a small amount of labeled samples in reality. To solve the problem of limited labeled samples, a semi-supervised road centerline extraction is proposed, which incorporates high-level feature selection, Markov random field (MRF), and ridge transversal method. The proposed road extraction approach consists of three steps: multiple features extraction, semi-supervised road area extraction, and road centerlines extraction. To get more abstract and discriminative high-level features, we apply multiple-feature adaptive sparse representation in mid-level features in different views generated by different prototype sets. To obtain an accurate road area result, we combine the feature learning framework with MRF. Then, we integrate Gabor filters and nonmaxima suppression with the ridge transversal method to extract centerlines. It is verified the proposed method achieves comparable performance with the state-of-the-art methods in terms of visual and quantitative aspects.

The studies on signed networks have received a great attention due to their capabilities on presenting conflicting relationships, which reflect the potential conflicts and tensions of complex systems. To further understand those conflicts and tensions, many methods have been proposed for computing the global structural balance (GSB) of signed networks, which aim to discover the most balanced state of the networks with the least number of unbalanced links. However, most of them request full access to all information (structures, signs, and balance states) of links, which are usually sensitive and private. In this article, we propose a privacy-preserving GSB computation (PGSBC) framework, which aims to compute the GSB while preserving the privacy of the networks. The PGSBC first protects the sensitive information (structures, signs, and balance states) of links by using encryption techniques (the homomorphic cryptosystem and the random disturbances) and then computes the GSB of the signed networks on the encrypted structures. In the PGSBC, a balance-aware energy function is adopted to evaluate the balance degree of a clustering, while a fast two-level greedy algorithm (called as HM-Louvain) is presented to discover the most balanced clustering of signed networks. Simulation results on 11 LFR benchmark networks and 10 real signed networks show that the proposed framework can effectively compute the GSB of the networks while preserving the privacy of links’ sensitive information.

The robust controllability (RC) of a complex system tries to select a set of dominating entities for the functional control of this entire system without uncertain disturbances, and the research on RC will help to understand the system's underlying functions. In this article, we introduce the control cost in signed networks and present a cost-aware robust control (CRC) problem in this scenario. The aim of CRC is to minimize the cost to control a set of dominating nodes and transform a set of unbalanced links into balanced ones, such that the signed network can be robustly controlled without uncertain unbalanced factors (like nodes and links). To solve this problem, we first model CRC as a constrained combination optimization problem, and then present a memetic algorithm with some problem-specific knowledge (like the neighbors of nodes, the constraints of CRC, and the fast computation of the cost under each optimization) to solve this problem on signed networks. The extensive experiments on both real social and biological networks assess that our algorithm outperforms several state-of-the-art RC algorithms.

Evolutionary multitask optimization (EMTO) is a newly emerging research area in the field of evolutionary computation. It investigates how to solve multiple optimization problems (tasks) at the same time via evolutionary algorithms (EAs) to improve on the performance of solving each task independently, assuming if some component tasks are related then the useful knowledge (e.g., promising candidate solutions) acquired during the process of solving one task may assist in (and also benefit from) solving the other tasks. In EMTO, task relatedness is typically unknown in advance and needs to be captured via EA's population. Since the population of an EA can only cover a subregion of the solution space and keeps evolving during the search, thus captured task relatedness is local and dynamic. The multifactorial EA (MFEA) is one of the most representative EMTO techniques, inspired by the bio-cultural model of multifactorial inheritance, which transmits both biological and cultural traits from the parents to the offspring. MFEA has succeeded in solving various multitask optimization (MTO) problems. However, the intensity of knowledge transfer in MFEA is determined via its algorithmic configuration without considering the degree of task relatedness, which may prevent the effective sharing and utilization of the useful knowledge acquired in related tasks. To address this issue, we propose a self-regulated EMTO (SREMTO) algorithm to automatically adapt the intensity of cross-task knowledge transfer to different and varying degrees of relatedness between different tasks as the search proceeds so that the useful knowledge in common for solving related tasks can be captured, shared, and utilized to a great extent. We compare SREMTO with MFEA and its variants as well as the single-task optimization counterpart of SREMTO on two MTO test suites, which demonstrates the superiority of SREMTO.

Unsupervised change detection based on supervised or semisupervised classifiers has achieved strong adaptability and robustness to obtain satisfactory change detection results. However, these methods suffer from an issue that it is hard to collect reliable training samples in an unsupervised manner. In this article, a group self-paced learning (GSPL) framework is proposed to mine the reliable training samples. In the proposed method, each sample is assigned a weight to indicate its reliability. The proposed scheme is able to learn the weighted samples and update the weights iteratively in a self-paced manner to identify the reliable training samples. In the phase of updating weights, a grouping strategy is designed to avoid selecting training samples from homogeneous regions. Furthermore, a novel time-varying self-paced regularizer is proposed to automatically determine the learning scheme of self-paced learning. Finally, three classifiers, including SoftMax, backpropagation neural network, and support vector machine, are investigated under this proposed framework. Experiments on five change detection data sets demonstrate that the proposed framework can significantly outperform those state-of-art methods for change detection in terms of accuracy and robustness.

Hyperspectral imagery is always composed of mixed pixels because of the limited spatial resolution of a sensor and the macroscopic/microscopic mixture of distinct substances. The linear mixing model (LMM) is proven to be simple and effective in extensive literature when the macroscopic mixture dominates the mixing process. But when the photons undergo multiple reflections before reaching the sensor, the LMM becomes invalid. In this circumstance, the bilinear mixture model (Bi-LMM), which considers secondary reflections with a bilinear term, is a viable alternative. However, the bilinear term in most existing Bi-LMMs is constructed based on the pre-estimated endmembers, and thus, most Bi-LMMs focus mainly on the abundance estimation. This may lead to inaccurate estimation of endmembers and abundances for a given hyperspectral image. In this article, we propose a multiobjective endmember extraction (Bi-MoEE) method within the bilinear mixture paradigm, which considers each secondary reflection as a virtual endmember. Then, Bi-MoEE selects real and virtual endmembers from an extended spectral library consisting of a standard spectral library and their virtual products. By imposing some intuitive constraints, the solution space is greatly reduced, and the multipoint crossover and restricted bit-flip mutation operators are specially designed. Finally, Bi-MoEE can efficiently obtain a set of tradeoff solutions by minimizing the unmixing residuals and the number of selected endmembers, and automatically determine the optimal solution with multiobjective decision-making techniques. Compared with some advanced endmember extraction methods, the proposed Bi-MoEE does not need to know the number of real endmembers. In addition, the time efficiency of Bi-MoEE is mainly related to the image size and the algorithmic parameters, and has little to do with the size of spectral library, thus facilitating the practical implementation of Bi-MoEE with regard to the oversized spectral library. The experiments on synthetic and real data sets demonstrated the excellent performance of Bi-MoEE.

The rapid development of remote sensing technology has enabled the acquisition of very high spatial resolution (VHR) multitemporal images in Earth observation. However, how to effectively exploit these existing data to accurately monitor land surface changes is still a challenging task. In this article, we propose an unsupervised scale-driven change detection (CD) framework for VHR images by jointly analyzing the spatial-spectral change information, which combines the advantages of deep feature learning and multiscale decision fusion. First, a well pretrained deep fully convolutional network (FCN) is used to automatically extract the deep spatial context information from the acquired images. Then, the uncertainty analysis incorporating the deep spatial feature and the image spectral feature is implemented to generate a pseudobinary change map. On this basis, it is easy to choose suitable samples to train an excellent support vector machine (SVM) classifier, thus detecting changes occurred on the ground. In addition, the multiscale superpixel segmentation technique is introduced to make full use of the spatial structural information, which takes an image-object as the basic analysis unit. Finally, a robust binary change map with high detection precision can be achieved by merging the CD results obtained at different scales. The impressive experimental results on four real data sets demonstrate the effectiveness and flexibility of the proposed framework.

The traditional art of Chinese calligraphy, reflecting the wisdom of the grass-roots community, is the soul of Chinese culture. Just like many other types of craftsmanship, it is part of the historical heritage and is worth conserving, from generation to generation. Since the movements of an ink brush are in a 3D style when Chinese calligraphy is written, they embody “The Power of Beauty,” comprising various reflectance properties and rough-surface geometry. To truly understand the powerful significance and beauty of the art of Chinese calligraphy, in this paper, a 3D calligraphy reconstruction method, based on Photometric Stereo, is designed to capture the detailed appearance of the calligraphy's 3D surface geometry. For assessment, an Iterative Closest Point (ICP) algorithm is applied for registration of 3D intrinsic shapes between the Chinese calligraphy and the calligraphy fans' handwriting. Through matching these two sets of calligraphy characters, the designed system can give a score to the handwriting of a user. Experiments have been performed on Chinese calligraphy from different historical dynasties to evaluate the effectiveness of the proposed scheme, and experimental results show that the developed system is useful and provides a convenient method of calligraphy appreciation and assessment.

Image change detection detects the regions of change in multiple images of the same scene taken at different times, which plays a crucial role in many applications. The two most popular image change detection techniques are as follows: pixel-based methods heavily rely on accurate image coregistration while object-based approaches can tolerate coregistration errors to some extent but are sensitive to image segmentation or classification errors. To address these issues, we propose an unsupervised image change detection approach based on a novel bipartite differential neural network (BDNN). The BDNN is a deep neural network with two input ends, which can extract the holistic features from the unchanged regions in the two input images, where two learnable change disguise maps (CDMs) are used to disguise the changed regions in the two input images, respectively, and thus demarcate the unchanged regions therein. The network parameters and CDMs will be learned by optimizing an objective function, which combines a loss function defined as the likelihood of the given input image pair over all possible input image pairs and two constraints imposed on CDMs. Compared with the pixel-based and object-based techniques, the BDNN is less sensitive to inaccurate image coregistration and does not involve image segmentation or classification. In fact, it can even skip over coregistration if the degree of transformation (due to the different view angles and/or positions of the camera) between the two input images is not that large. We compare the proposed approach with several state-of-the-art image change detection methods on various homogeneous and heterogeneous image pairs with and without coregistration. The results demonstrate the superiority of the proposed approach.

In this article, a new deep neural network based on sparse filtering and manifold regularization (DSMR) is proposed for feature extraction and classification of polarimetric synthetic aperture radar (PolSAR) data. DSMR uses a novel deep neural network (DNN) to automatically learn features from raw SAR data. During preprocessing, the spatial information between pixels on PolSAR images is exploited to weight each data sample. Then, in the pretraining and fine-tuning, DSMR uses the population sparsity and the lifetime sparsity (dual sparsity) to learn the global features and preserves the local structure of data by neighborhood-based manifold regularization. The dual sparsity only needs to tune a few parameters, and the manifold regularization cuts down the number of training samples. Experimental results on synthesized and real PolSAR data sets from different SAR systems show that DSMR can improve classification accuracy compared with conventional DNNs, even for data sets with a large angle of incidence.

Fuzzy clustering algorithm based on multiobjective optimization can achieve accurate and comprehensive clustering results. However, the estimation of objective values for this multiobjective optimization problem (MOP) might be expensive. Offspring's selection driven by simple evaluation is time consuming. Therefore, we integrate regression techniques to determine the superiority of the offspring solutions in the evolution process. However, it suffers from an issue that it is hard to collect reliable samples to train such a robust regression model. In this paper, an evolutionary multiobjective fuzzy clustering method via self-paced learning is proposed for change detection. In the proposed method, the self-paced learning process is implemented to collect reliable training samples for training a robust regression model, which can help to select promising offspring solutions from the candidate solutions for MOP. Experiments on three remote sensing image datasets demonstrate that the proposed method can significantly outperform those state-of-art methods for change detection in terms of accuracy and robustness.

Currently, the most of multitask evolutionary algorithms views multiple tasks as factors influencing the evolution of individuals. However, this consideration causes difficulty to assign fitness to individuals, because an individual which performs well on one task can have a bad performance on another task. To avoid this difficulty, this paper proposes a novel multipopulation technique for multitask optimization (MPMTO). The novelty of MPMTO is that it can solve the multiple tasks via a simple and straightforward method by corresponding each population to a task. By this way, the fitness assignment issue can be addressed by just assigning the objective value of the corresponding task to individuals. MPMTO is a general technique so that existing population-based optimization algorithms can be used in each population. This paper uses differential evolutionary algorithm in each population and develops a multipopulation multitask differential evolutionary optimization (mMTDE) based on the proposed multipopulation technique. mMTDE features that each population can use the other populations as the additional knowledge source to create an overlapping population, allowing the populations share information. By this way, the population can improve the efficacy and accuracy of solving multiple tasks. Moreover, the successful inter-task offspring can immigrate back to the corresponding population to fully utilize the inter-task knowledge. We have compared the proposed method with other state-of-the-art methods on benchmark multitask problems. The experimental results show the superiority of the proposed method which could utilizes efficiently the searching knowledge of multiple tasks.

There is a ruling maxim in deep learning land, bigger is better. However, bigger neural network provides higher performance but also expensive computation, memory and energy. The simplified model which preserves the accuracy of original network arouses a growing interest. A simple yet efficient method is pruning, which cuts off unimportant synapses and neurons. Therefore, it is crucial to identify important parts from the given numerous connections. In this paper, we use the evolutionary pruning method to simplify the structure of deep neural networks. A multi-objective neural networks pruning model which balances the accuracy and the sparse ratio of networks is proposed and we solve this model with particle swarm optimization (PSO) method. Furthermore, we fine-tune the network which is obtained by pruning to obtain better pruning result. The framework of alternate pruning and fine-tuning operations is used to achieve more prominent pruning effect. In experimental studies, we prune LeNet on MNIST and shallow VGGNet on CIFAR-10. Experimental results demonstrate that our method could prune over 80% weights in general with no loss of accuracy.

Evolutionary multi-task optimization (EMTO) studies on how to simultaneously solve multiple optimization problems, so-called component problems, via evolutionary algorithms, which has drawn much attention in the field of evolutionary computation. Knowledge transfer across multiple optimization problems (being solved) is the key to make EMTO to outperform traditional optimization paradigms. In this work, we propose a simple and effective knowledge transfer strategy which utilizes the best solution found so far for one problem to assist in solving the other problems during the optimization process. This strategy is based on random replacement. It does not introduce extra computational cost in terms of objective function evaluations for solving each component problem. However, it helps to improve optimization effectiveness and efficiency, compared to solving each component problem in a standalone way. This light-weight knowledge transfer strategy is implemented via differential evolution within a multi-population based EMTO paradigm, leading to a differential evolutionary multi-task optimization (DEMTO) algorithm. Experiments are conducted on the CEC'2017 competition test bed to compare the proposed DEMTO algorithm with five state-of-the-art EMTO algorithms, which demonstrate the superiority of DEMTO.

Due to the distinct statistical properties in cross-sensor images, change detection in heterogeneous images is much more challenging than in homogeneous images. In this letter, we adopt a conditional generative adversarial network (cGAN) to transform the heterogeneous synthetic aperture radar (SAR) and optical images into some space where their information has a more consistent representation, making the direct comparison feasible. Our proposed framework contains a cGAN-based translation network that aims to translate the optical image with the SAR image as a target, and an approximation network that approximates the SAR image to the translated one by reducing their pixelwise difference. The two networks are updated alternately and when they are both trained well, the two translated and approximated images can be considered as homogeneous, from which the final change map can be acquired by direct comparison. Theoretical analysis and experimental results demonstrate the effectiveness and robustness of the proposed framework.

Non-negative matrix factorization (NMF) is becoming increasingly popular in many research fields due to its particular properties of semantic interpretability and part-based representation. Sparseness constraints are usually imposed on the NMF problems in order to achieve potential features and sparse representation. These constrained NMF problems are usually reformulated as regularization models to solve conveniently. However, the regularization parameters in the regularization model are difficult to tune and the frequently used sparse-inducing terms in the regularization model generally have bias effects on the induced matrix and need an extra restricted isometry property (RIP). This paper proposes a multiobjective sparse NMF paradigm which refrains from the regularization parameter issues, bias effects, and the RIP condition. A novel multiobjective memetic algorithm is also proposed to generate a set of solutions with diverse sparsity and high factorization accuracy. A masked projected gradient local search scheme is specially designed to accelerate the convergence rate. In addition, a priori knowledge is also integrated in the algorithm to reduce the computational time in discovering our interested region in the objective space. The experimental results show that the proposed paradigm has better performance than some regularization algorithms in producing solutions with different degrees of sparsity as well as high factorization accuracy, which are favorable for making the final decisions.

Complex systems are often characterized by complex networks with links and entities. However, in many complex systems such as protein-protein interaction networks, recommender systems, and online communities, their links are hard to reveal directly, but they can be inaccurately observed by multiple data collection platforms or by a data collection platform at different times. Then, the links of the systems are inferred by the integration of the collected observations. As those data collection platforms are usually distributed over a large area and in different fields, their observations are unreliable and sensitive to the potential structures of the systems. In this paper, we consider the link inference problem in network data with community structures, in which the reliability of data collection platforms is unknown a priori and the link errors and reliability of platforms' observations are heterogeneous to the underlying community structures of the systems. We propose an expectation maximization algorithm for link inference in a network system with community structures (EMLIC). The EMLIC algorithm is also used to infer the link errors and reliability of platforms' observations in different communities. Experimental results on both synthetic data and eight real-world network data demonstrate that our algorithm is able to achieve lower link errors than the existing reliable link inference algorithms when the network data have community structures.

Self-paced learning (SPL) is a recently proposed paradigm to imitate the learning process of humans/animals. SPL involves easier samples into training at first and then gradually takes more complex ones into consideration. Current SPL regimes incorporate a self-paced (SP) regularizer into the learning objective with a gradually increasing pace parameter. Therefore, it is difficult to obtain the solution path of the SPL regime and determine where to optimally stop this increasing process. In this paper, a multiobjective SPL method is proposed to optimize the loss function and the SP regularizer simultaneously. A decomposition-based multiobjective particle swarm optimization algorithm is used to simultaneously optimize the two objectives for obtaining the solutions. In the proposed method, a polynomial soft weighting regularizer is proposed to penalize the loss. Theoretical studies are conducted to show that the previous regularizers are roughly particular cases of the proposed polynomial soft weighting regularizer family. Then an implicit decomposition method is proposed to search the solutions with respect to the sample number involved into training. A set of solutions can be obtained by the proposed method and naturally constitute the solution path of the SPL regime. Then a satisfactory solution can be naturally obtained from these solutions by utilizing some effective tools in evolutionary multiobjective optimization. Experiments on matrix factorization and classification problems demonstrate the effectiveness of the proposed technique.

Evolutionary multitasking is a recently proposed paradigm to simultaneously solve multiple tasks using a single population. Most of the existing evolutionary multitasking algorithms treat all tasks equally and then assign the same amount of resources to each task. However, when the resources are limited, it is difficult for some tasks to converge to acceptable solutions. This paper aims at investigating the resource allocation in the multitasking environment to efficiently utilize the restrictive resources. In this paper, we design a novel multitask evolutionary algorithm with an online dynamic resource allocation strategy. Specifically, the proposed dynamic resource allocation strategy allocates resources to each task adaptively according to the requirements of tasks. We also design an adaptive method to control the resources invested into cross-domain searching. The proposed algorithm is able to allocate the computational resources dynamically according to the computational complexities of tasks. The experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-art algorithms on benchmark problems of multitask optimization.

Real-world applications typically have multiple sparse reconstruction tasks to be optimized. In order to exploit the similar sparsity pattern between different tasks, this paper establishes an evolutionary multitasking framework to simultaneously optimize multiple sparse reconstruction tasks using a single population. In the proposed method, the evolutionary algorithm aims to search the locations of nonzero components or rows instead of searching sparse vector or matrix directly. Then the within-task and between-task genetic transfer operators are employed to reinforce the exchange of genetic material belonging to the same or different tasks. The proposed method can solve multiple measurement vector problems efficiently because the length of decision vector is independent of the number of measurement vectors. Finally, a case study on hyperspectral image unmixing is investigated in an evolutionary multitasking setting. It is natural to consider a sparse unmixing problem in a homogeneous region as a task. Experiments on signal reconstruction and hyperspectral image unmixing demonstrate the effectiveness of the proposed multitasking framework for sparse reconstruction.

This paper suggests a novel clustering-based evolutionary algorithm for many-objective optimization problems. Its main idea is to classify the population into a number of clusters, which is expected to solve the difficulty of balancing convergence and diversity in high-dimensional objective space. The individuals showing high similarities on the vector angles are gathered into the same cluster, such that the population’s distribution can be well portrayed by the clusters. To efficiently find these clusters, partitional clustering is first used to classify the union population into${m}$main clusters based on the${m}$axis vectors (${m}$is the number of objectives), and then hierarchical clustering is further run on these${m}$main clusters to get${N}$final clusters (${N}$is the population size and${N>m}$). At last, in environmental selection, one individual from each of${N}$clusters closest to the axis vectors is selected to maintain diversity, while one individual from each of the other clusters is preferred by a simple convergence indicator to ensure convergence. When tackling some well-known test problems with 5–15 objectives, extensive experiments validate the superiority of our algorithm over six competitive many-objective EAs, especially on problems with incomplete and irregular Pareto-optimal fronts.

Feature extraction (FE) is a crucial research area in hyperspectral image (HSI) processing. Recently, due to the powerful ability of deep learning (DL) to extract spatial and spectral features, DL-based FE methods have shown great potentials for HSI processing. However, most of the DL-based FE methods are supervised, and the training of them suffers from the absence of labeled samples in HSIs severely. The training issue of supervised DL-based FE methods limits their application on HSI processing. To address this issue, in this paper, a novel modified generative adversarial network (GAN) is proposed to train a DL-based feature extractor without supervision. The designed GAN consists of two components, which are a generator and a discriminator. The generator can focus on the learning of real probability distributions of data sets and the discriminator can extract spatial-spectral features with superior invariance effectively. In order to learn upsampling and downsampling strategies adaptively during FE, the proposed generator and discriminator are designed based on a fully deconvolutional subnetwork and a fully convolutional subnetwork, respectively. Moreover, a novel min-max cost function is designed for training the proposed GAN in an end-to-end fashion without supervision, by utilizing the zero-sum game relationship between the generator and discriminator. Besides, the proposed modified GAN replaces the original Jensen-Shannon divergence with the Wasserstein distance, aiming to mitigate the unstability and difficulty of the training of GAN frameworks. Experimental results on three real data sets validate the effectiveness of the proposed method.

With the rapid increase of remote sensing images in temporal, spectral, and spatial resolutions, it is urgent to develop effective techniques for joint interpretation of spatial-temporal images. Multitype change detection (CD) is a significant research topic in multitemporal remote sensing image analysis, and its core is to effectively measure the difference degree and represent the difference among the multitemporal images. In this paper, we propose a novel difference representation learning (DRL) network and present an unsupervised learning framework for multitype CD task. Deep neural networks work well in representation learning but rely too much on labeled data, while clustering is a widely used classification technique free from supervision. However, the distribution of real remote sensing data is often not very friendly for clustering. To better highlight the changes and distinguish different types of changes, we combine difference measurement, DRL, and unsupervised clustering into a unified model, which can be driven to learn Gaussian-distributed and discriminative difference representations for nonchange and different types of changes. Furthermore, the proposed model is extended into an iterative framework to imitate the bottom-up aggregative clustering procedure, in which similar change types are gradually merged into the same classes. At the same time, the training samples are updated and reused to ensure that it converges to a stable solution. The experimental studies on four pairs of multispectral data sets demonstrate the effectiveness and superiority of the proposed model on multitype CD.

In sparse hyperspectral unmixing, regularization methods inevitably suffer from the "decision ahead of solution" issue concerning the regularization parameter, which is not conducive to practical applications. To settle this issue, a two-phase multiobjective sparse unmixing (Tp-MoSU) approach has been proposed recently. However, Tp-MoSU has limited performance on high noise data and uses little spatial-contextual information in estimating abundances. To address the first problem, a tri-objective optimization model is established for each of the two phases to model mixed additive noise automatically. To address the second problem, a dual spatial exploiting objective is specially designed in the second phase to exploit similarity among adjacent pixels, which can improve the quality of estimated abundances. In addition, the memetic based evolutionary algorithms are elaborately modified for each of the two phases for better convergence. The experimental results on several representative data sets demonstrate that the proposed method performs better than Tp-MoSU in both of the two phases and completely better than some advanced regularization algorithms in abundance estimation under mixed additive noise.

The goal of network representation learning is to embed nodes so as to encode the proximity structures of a graph into a continuous low-dimensional feature space. In this paper, we propose a novel algorithm called node2hash based on feature hashing for generating node embeddings. This approach follows the encoder-decoder framework. There are two main mapping functions in this framework. The first is an encoder to map each node into high-dimensional vectors. The second is a decoder to hash these vectors into a lower dimensional feature space. More specifically, we firstly derive a proximity measurement called expected distance as target which combines position distribution and co-occurrence statistics of nodes over random walks so as to build a proximity matrix, then introduce a set of T different hash functions into feature hashing to generate uniformly distributed vector representations of nodes from the proximity matrix. Compared with the existing state-of-the-art network representation learning approaches, node2hash shows a competitive performance on multi-class node classification and link prediction tasks on three real-world networks from various domains.

Image registration is an important preprocessing step for many remote sensing image processing applications, and its result will affect the performance of the follow-up procedures. Establishing reliable matches is a key issue in point matching-based image registration. Due to the significant intensity mapping difference between remote sensing images, it may be difficult to find enough correct matches from the tentative matches. In this letter, particle swarm optimization (PSO) sample consensus algorithm is proposed for remote sensing image registration. Different from random sample consensus (RANSAC) algorithm, the proposed method directly samples the modal transformation parameter rather than randomly selecting tentative matches. Thus, the proposed method is less sensitive to the correct rate than RANSAC, and it has the ability to handle lower correct rate and more matches. Meanwhile, PSO is utilized to optimize parameter as its efficiency. The proposed method is tested on several multisensor remote sensing image pairs. The experimental results indicate that the proposed method yields a better registration performance in terms of both the number of correct matches and aligning accuracy.

With the rapid development of remote sensing technology, how to accurately detect changes that have occurred on the land surface has been a critical task, particularly when images come from different satellite sensors. In this letter, we propose an unsupervised change detection method for heterogeneous synthetic aperture radar (SAR) and optical images based on the logarithmic transformation feature learning framework. First, the logarithmic transformation is applied to the SAR image that aims to achieve similar statistical distribution properties as the optical image. Then, high-level feature representations can be learned from the transformed image pair via joint feature extraction, which are used to select reliable samples for training a neural network classifier. When it is trained well, a robust change map can be obtained, thus identifying changed regions accurately. The experimental results on three real heterogeneous data sets demonstrate the effectiveness and superiority of the proposed method compared with other existing state-of-the-art approaches.

Molecular interactions data increase exponentially with the advance of biotechnology. This makes it possible and necessary to comparatively analyze the different data at a network level. Global network alignment is an important network comparison approach to identify conserved subnetworks and get insight into evolutionary relationship across species. Network alignment which is analogous to subgraph isomorphism is known to be an NP-hard problem. In this paper, we introduce a novel heuristic Particle-Swarm-Optimization based Network Aligner (PSONA), which optimizes a weighted global alignment model considering both protein sequence similarity and interaction conservations. The particle statuses and status updating rules are redefined in a discrete form by using permutation. A seed-and-extend strategy is employed to guide the searching for the superior alignment. The proposed initialization method "seeds" matches with high sequence similarity into the alignment, which guarantees the functional coherence of the mapping nodes. A greedy local search method is designed as the "extension" procedure to iteratively optimize the edge conservations. PSONA is compared with several state-of-art methods on ten network pairs combined by five species. The experimental results demonstrate that the proposed aligner can map the proteins with high functional coherence and can be used as a booster to effectively refine the well-studied aligners.

With the sparse unmixing becoming increasingly popular recently, some advanced regularization algorithms have been proposed for settling this problem. However, they are limited by their “decision ahead of solution” attribute, i.e., the regularization parameters must be preset before the solution is obtained. In this paper, the sparse unmixing problem is first formulated as a two-phase multiobjective problem. The first phase simultaneously minimizes the unmixing residuals and the number of estimated endmembers for automatically finding the real active endmembers from the spectral library. A decomposition-based endmember selection algorithm considering the gene exchange in the population is specially designed for better and quicker search of the decision space. This algorithm can obtain a set of nondominated solutions for better decision of the active endmembers, which are important for the subsequent calculation of the abundance matrix. The second phase concurrently minimizes the unmixing residuals and the total variation term for estimating a preferable abundance matrix. A local search strategy based on the multiplicative update rule is designed in the evolution process for better approximation of the Pareto front. The experimental results on the synthetic as well as the real data reveal that the proposed framework has a better performance in finding the real active endmembers and estimating their corresponding abundances than some advanced regularization algorithms.

This paper focuses on the connecting structure of deep neural networks and proposes a layerwise structure learning method based on multiobjective optimization. A model with better generalization can be obtained by reducing the connecting parameters in deep networks. The aim is to find the optimal structure with high representation ability and better generalization for each layer. Then, the visible data are modeled with respect to structure based on the products of experts. In order to mitigate the difficulty of estimating the denominator in PoE, the denominator is simplified and taken as another objective, i.e., the connecting sparsity. Moreover, for the consideration of the contradictory nature between the representation ability and the network connecting sparsity, the multiobjective model is established. An improved multiobjective evolutionary algorithm is used to solve this model. Two tricks are designed to decrease the computational cost according to the properties of input data. The experiments on single-layer level, hierarchical level, and application level demonstrate the effectiveness of the proposed algorithm, and the learned structures can improve the performance of deep neural networks.

We propose an unsupervised deep convolutional coupling network for change detection based on two heterogeneous images acquired by optical sensors and radars on different dates. Most existing change detection methods are based on homogeneous images. Due to the complementary properties of optical and radar sensors, there is an increasing interest in change detection based on heterogeneous images. The proposed network is symmetric with each side consisting of one convolutional layer and several coupling layers. The two input images connected with the two sides of the network, respectively, are transformed into a feature space where their feature representations become more consistent. In this feature space, the different map is calculated, which then leads to the ultimate detection map by applying a thresholding algorithm. The network parameters are learned by optimizing a coupling function. The learning process is unsupervised, which is different from most existing change detection methods based on heterogeneous images. Experimental results on both homogenous and heterogeneous images demonstrate the promising performance of the proposed network compared with several existing approaches.

Nowadays, a lot of people possess accounts on multiple online social networks, e.g., Facebook and Twitter. These networks are overlapped, but the correspondences between their users are not explicitly given. Mapping common users across these social networks will be beneficial for applications such as cross-network recommendation. In recent years, a lot of mapping algorithms have been proposed which exploited social and/or profile relations between users from different networks. However, there is still a lack of unified mapping framework which can well exploit high-order relational information in both social structures and profiles. In this paper, we propose a unified hypergraph learning framework named unified manifold alignment on hypergraph (UMAH) for this task. UMAH models social structures and user profile relations in a unified hypergraph where the relative weights of profile hyperedges are determined automatically. Given a set of training user correspondences, a common subspace is learned by preserving the hypergraph structure as well as the correspondence relations of labeled users. UMAH intrinsically performs semisupervised manifold alignment with profile information for calibration. For a target user in one network, UMAH ranks all the users in the other network by their probabilities of being the corresponding user (measured by similarity in the subspace). In experiments, we evaluate UMAH on three real world data sets and compare it to state-of-art baseline methods. Experimental results have demonstrated the effectiveness of UMAH in mapping users across networks.

This paper presents a novel neuron learning machine (NLM) which can extract hierarchical features from data. We focus on the single-layer neural network architecture and propose to model the network based on the Hebbian learning rule. Hebbian learning rule describes how synaptic weight changes with the activations of presynaptic and postsynaptic neurons. We model the learning rule as the objective function by considering the simplicity of the network and stability of solutions. We make a hypothesis and introduce a correlation based constraint according to the hypothesis. We find that this biologically inspired model has the ability of learning useful features from the perspectives of retaining abstract information. NLM can also be stacked to learn hierarchical features and reformulated into convolutional version to extract features from 2-dimensional data.

Image change detection is to recognize the changes between two images that are taken over the same scene but at different times, which has been applied broadly in many fields. Fuzzy clustering is a frequently-used technique for unsupervised change detection. However, traditional fuzzy clustering algorithms are easy to be trapped into a local optimum due to the limits of their optimization processes. To tackle the problem, a novel differential evolution algorithm with an automatically learning selection strategy is proposed in this paper. Different from the selection rules of classical differential evolution algorithm, this method firstly pre-classifies all original individuals and trial individuals according to the scope of the individual fitness at each generation, which will preliminarily determine whether they are selected for the next generation. Secondly, in order to increase the diversity of the population, we choose a few individuals from the non-selected population with a low probability into selected ones. Finally, the samples including partial individuals from the selected and non-selected lists are used to train the neural networks that will learn the selection strategy. This method will learn different selection strategies in every generation, which will significantly accelerate the convergence speed. The proposed change detection method, combining fuzzy clustering with newly designed differential evolution algorithm, show excellent performance. Experiments conducted on Synthetic Aperture Radar images have demonstrated the superiority of the proposed method.

Neural networks are currently popular learning models to represent and analyze data. We address two issues about that in this paper. On the one hand, the parameters between neurons are often restricted to be constants, which greatly limits the learning ability and reduces the robustness of the neural network. For that, it is necessary to make the parameters fuzzy. In this paper, we introduce the fuzzy set theory to neural networks where the parameters are expressed by fuzzy numbers. Meanwhile, the loss term and sparsity of the network become fuzzy. On the other hand, a user-defined weighting parameter need to be determined to keep the trade-off between the fuzzy loss term and fuzzy sparsity. In order to solve the two issues simultaneously, the main contribution of this paper is to combine fuzzy set theory with multi-objective sparsity to apply to neural networks, for the first time, and propose a fuzzy multi-objective sparse feature learning (FMSFL) model, where a multi-objective optimization model is established, and reconstruction error and sparsity of fuzzy model are considered as two objectives. In the experiments, we demonstrate the effectiveness of our model, and both learning capability and robustness of the neural networks based on the proposed model are improved.

Endmember extraction is a critical step of spectral unmixing. In this paper, a novel endmember extraction algorithm based on evolutionary multi-objective optimization is proposed for hyperspectral remote sensing images. In the proposed method, endmember extraction is modeled as a multi-objective optimization problem. Then the root mean square error between the original image and its remixed image and the number of endmembers are chosen as two conflicting objective functions, which are simultaneously optimized by particle swarm optimization algorithm to find the best tradeoff solutions. In order to promote diversity and speed up the convergence of the algorithm, a new particle status updating strategy and a novel method for selecting leaders are designed. The experimental results on both simulated and real hyperspectral remote sensing images confirm the performance of the proposed approach over some existing methods.

The detection of shared community structure in multilayer network is an interesting and important issue that has attracted many researches. Traditional methods for community detection of single layer networks are not suitable for that of multilayer networks. In a previous work, the authors modeled the community discovery problem in multilayer network as a multiobjective one and devised a genetic algorithm to carry out it. In this paper, based on their model, we propose an improved multiobjective evolutionary approach MOEA-MultiNet for community detection in multilayer networks. The proposed MOEA-MultiNet is based on the framework of NSGA-II which employs the string-based representation scheme and synthesizes the genetic operation and local search to perform individual refinement. Experimental results on two real-world networks both demonstrate the ability and efficiency of the proposed MOEA-MultiNet in detecting community structure in multilayer networks.

Evolutionary multi-tasking is a novel concept where algorithms utilize the implicit parallelism of population-based search to solve several tasks efficiently. In last decades, multi-task learning, which harnesses the underlying similarity of the learning tasks, has proved efficient in many applications. Extreme learning machine is a distinctive learning algorithm for feed-forward neural networks. Because of its similarity and low computational complexity comparing with the convenient neural network training algorithms, it has been used in many cases of data analyses. In this paper, a modular training technique by employing evolutionary multi-task paradigm is used to evolve the modular topologies of extreme learning machine. Though, extreme learning machine is much faster than the convenient gradient-based method, it needs more hidden neurons due to the random determination of input weights. In proposed method, we combine the evolutionary extreme learning machine and multi-task modular training. Each task is defined by an evolutionary extreme learning machine with different number of hidden neurons. This method produces a modular extreme learning machine which needs less number of hidden units and could be effective even if some hidden neurons and connections are removed. Experiment results show effectiveness and generalization of the proposed method for benchmark classification problems.

The aim of influence maximization problem is to mine a small set of influential individuals in a complex network which could reach the maximum influence spread. In this paper, an efficient fitness function based on local influence is designed to estimate the influence spread. Then, we propose a discrete particle swarm optimization based algorithm to find the final set with the maximum value of the fitness function. In the proposed algorithm, discrete position and velocity are redefined and problem-specific update rules are designed. In order to accelerate the convergence, we introduce a degree-based population initialization method and a mutation learning based local search strategy. Experimental results compared with four comparison algorithms show that our proposed algorithm is able to efficiently find good-quality solutions.

Band selection is a crucial preprocessing step for hyperspectral image classification, which is a classic feature selection method. Feature selection is designed to select feature subsets to represent the whole feature space. For feature selection, two crucial issues need to be handled: preserving information and redundancy reducing. In this paper, a novel feature selection method for hyperspectral image classification is proposed, which is based on a newly designed memetic algorithm. In the proposed method, a suitable objective function is designed, which can measure the contained crucial information and redundancy information in the selected feature subsets. To optimize this objective function efficiently, a novel memetic algorithm is designed. The genetic operator and local search strategy are newly designed according to the characteristic of hyperspectral images. Experiments are implemented on three real data sets compared with some state of arts. The experimental results show that the proposed method can obtain stable and superior feature subsets for classification.

Convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks. In order to distinguish the reliable data from the noisy and confusing data, we improve CNNs with self-paced learning (SPL) for enhancing the learning robustness of CNNs. In the proposed self-paced convolutional network (SPCN), each sample is assigned to a weight to reflect the easiness of the sample. Then a dynamic self-paced function is incorporated into the leaning objective of CNN to jointly learn the parameters of CNN and the latent weight variable. SPCN learns the samples from easy to complex and the sample weights can dynamically control the learning rates for converging to better values. To gain more insights of SPCN, theoretical studies are conducted to show that SPCN converges to a stationary solution and is robust to the noisy and confusing data. Experimental results on MNIST and rectangles datasets demonstrate that the proposed method outperforms baseline methods.

This paper presents to model the Hebb learning rule and proposes a neuron learning machine (NLM). Hebb learning rule describes the plasticity of the connection between presynaptic and postsynaptic neurons and it is unsupervised itself. It formulates the updating gradient of the connecting weight in artificial neural networks. In this paper, we construct an objective function via modeling the Hebb rule. We make a hypothesis to simplify the model and introduce a correlation based constraint according to the hypothesis and stability of solutions. By analysis from the perspectives of maintaining abstract information and increasing the energy based probability of observed data, we find that this biologically inspired model has the capability of learning useful features. NLM can also be stacked to learn hierarchical features and reformulated into convolutional version to extract features from 2-dimensional data. Experiments on single-layer and deep networks demonstrate the effectiveness of NLM in unsupervised feature learning.

Change detection and analysis (CDA) is an important research topic in the joint interpretation of spatial-temporal remote sensing images. The core of CDA is to effectively represent the difference and measure the difference degree between bi-temporal images. In this paper, we propose a novel difference representation learning network (DRLnet) and an effective optimization framework without any supervision. Difference measurement, difference representation learning and unsupervised clustering are combined as a single model, i.e., DRLnet, which is driven to learn clustering-friendly and discriminative difference representations (DRs) for different types of changes. Further, DRLnet is extended into a recurrent learning framework to update and reuse limited training samples and prevent the semantic gaps caused by the saltation in the number of change types from over-clustering stage to the desired one. Experimental results identify the effectiveness of the proposed framework.

Summary Immunodominance clone selection algorithm (ICSA) is a robust and effective metaheuristic method for feature selection problem. However, ICSA is usually slow in finding the optimal solution. In this paper, we propose a parallel immunodominance clone selection algorithm (PICSA) on Graphics Processing Unit (GPU) to improve the speedup of ICSA for feature selection problem. The parallel program can considerably accelerate the feature selection operator. The immunodominance operator, which efficiently connects the local and global information, makes the algorithm able to jump out of the local optimum easily and obtain the global optimum. When comparing with other parallel languages, Open Computing Language (OpenCL) has advantages both in efficiency and portability. Therefore, we use OpenCL to implement this algorithm on Intel many integrated core and different GPU platforms. Experiment results obtained using high‐dimensional UCI machine learning and image texture datasets demonstrate that the PICSA algorithm allows one to achieve good acceleration ratio while maintaining similar classification accuracy to serial ICSA program. Besides, the OpenCL‐based implementation of PICSA shows good portability on many integrated core and different GPU platforms as well. Copyright © 2016 John Wiley & Sons, Ltd.

Change detection can be treated as a generative learning procedure, in which the connection between bitemporal images and the desired change map can be modeled as a generative one. In this letter, we propose an unsupervised change detection method based on generative adversarial networks (GANs), which has the ability of recovering the training data distribution from noise input. Here, the joint distribution of the two images to be detected is taken as input and an initial difference image (DI), generated by traditional change detection method such as change vector analysis, is used to provide prior knowledge for sampling the training data based on Bayesian theorem and GAN's min-max game theory. Through the continuous adversarial learning, the shared mapping function between the training data and their corresponding image patches can be built in GAN's generator, from which a better DI can be generated. Finally, an unsupervised clustering algorithm is used to analyze the better DI to obtain the desired binary change map. Theoretical analysis and experimental results demonstrate the effectiveness and robustness of the proposed method.

The scale-invariant feature transform algorithm and its many variants are widely used in feature-based remote sensing image registration. However, it may be difficult to find enough correct correspondences for remote image pairs in some cases that exhibit a significant difference in intensity mapping. In this letter, a new gradient definition is introduced to overcome the difference of image intensity between the remote image pairs. Then, an enhanced feature matching method by combining the position, scale, and orientation of each keypoint is introduced to increase the number of correct correspondences. The proposed algorithm is tested on multispectral and multisensor remote sensing images. The experimental results show that the proposed method improves the matching performance compared with several state-of-the-art methods in terms of the number of correct correspondences and aligning accuracy.

Due to the lack of label information and the intrinsic complexity of hyperspectral images (HSIs), unsupervised band selection is always one of the most challenging tasks in HSI processing. Fuzzy clustering is a promising technique for unsupervised band selection, which can partition unlabeled data into groups effectively. However, due to the limits of its optimization process, standard fuzzy clustering is sensitive to initialization and easy to be trapped in a local optimum. To address the limits, a novel unsupervised band selection method is proposed, combining fuzzy clustering with particle swarm optimization (PSO). A newly designed PSO algorithm is introduced to improve the performance of fuzzy clustering band selection. Moreover, a new strategy is designed to select representative cluster centers according to the characteristics of HSIs. The experimental results indicate that the proposed method has the ability to select high-quality band subsets with good and robust performance on HSI classification.

Sparse unmixing of hyperspectral data is an important technique aiming at estimating the fractional abundances of the end members. Traditional sparse unmixing is faced with the l0-norm problem which is an NP-hard problem. Sparse unmixing is inherently a multiobjective optimization problem. Most of the recent works combine cost functions into single one to construct an aggregate objective function, which involves weighted parameters that are sensitive to different data sets and difficult to tune. In this paper, a novel multiobjective cooperative coevolutionary algorithm is proposed to optimize the reconstruction term, the sparsity term and the total variation regularization term simultaneously. A problem-dependent cooperative coevolutionary strategy is designed because sparse unmixing encounters a large scale optimization problem. The proposed approach optimizes the nonconvex l0-norm problem directly and can find a better compromise between two or more competing cost function terms automatically. Experimental results on simulated and real hyperspectral data sets demonstrate the effectiveness of the proposed method.

The deployment of the airships plays a key role in maximizing the performance of the near space communication system. The main problem is how to strike a balance between the conflicting network speed and coverage for complex user distribution. In this paper, we propose a multiobjective deployment optimization model considering path loss, user demand, and inner structure. Under the framework of the multiobjective evolutionary algorithm (MOEA) based on decomposition (MOEA/D), we propose a similarity-based MOEA to optimize this problem. The proposed algorithm is motivated by the population's perception on the decision variable space. The proposed algorithm perceives the decision variable space by deploying airships to latent regions. The perceptions of different solutions are related by the similarity between their deployments and utilized differently by crossover and mutation. The proposed algorithm is tested on five designed problems compared with MOEA/D with the other popular reproduction operators. We also test the proposed scheme integrated with another two popular algorithms. The experimental results show that the similarity-based MOEA/D outperforms the other algorithms significantly in detecting hotspots, tracking multiple hotspots and safely deploying airships for most cases. The proposed scheme also works well with the other algorithms.

With the rapid technological development of various satellite sensors, high-resolution remotely sensed imagery has been an important source of data for change detection in land cover transition. However, it is still a challenging problem to effectively exploit the available spectral information to highlight changes. In this paper, we present a novel change detection framework for high-resolution remote sensing images, which incorporates superpixel-based change feature extraction and hierarchical difference representation learning by neural networks. First, highly homogenous and compact image superpixels are generated using superpixel segmentation, which makes these image blocks adhere well to image boundaries. Second, the change features are extracted to represent the difference information using spectrum, texture, and spatial features between the corresponding superpixels. Third, motivated by the fact that deep neural network has the ability to learn from data sets that have few labeled data, we use it to learn the semantic difference between the changed and unchanged pixels. The labeled data can be selected from the bitemporal multispectral images via a preclassification map generated in advance. And then, a neural network is built to learn the difference and classify the uncertain samples into changed or unchanged ones. Finally, a robust and high-contrast change detection result can be obtained from the network. The experimental results on the real data sets demonstrate its effectiveness, feasibility, and superiority of the proposed technique.

With the application requirement, the technique for change detection based on heterogeneous remote sensing images is paid more attention. However, detecting changes between two heterogeneous images is challenging as they cannot be compared in low-dimensional space. In this paper, we construct an approximately symmetric deep neural network with two sides containing the same number of coupled layers to transform the two images into the same feature space. The two images are connected with the two sides and transformed into the same feature space, in which their features are more discriminative and the difference image can be generated by comparing paired features pixel by pixel. The network is first built by stacked restricted Boltzmann machines, and then, the parameters are updated in a special way based on clustering. The special way, motivated by that two heterogeneous images share the same reality in unchanged areas and retain respective properties in changed areas, shrinks the distance between paired features transformed from unchanged positions, and enlarges the distance between paired features extracted from changed positions. It is achieved through introducing two types of labels and updating parameters by adaptively changed learning rate. This is different from the existing methods based on deep learning that just do operations on positions predicted to be unchanged and extract only one type of labels. The whole process is completely unsupervised without any priori knowledge. Besides, the method can also be applied to homogeneous images. We test our method on heterogeneous images and homogeneous images. The proposed method achieves quite high accuracy.

Current self-paced learning (SPL) regimes adopt the greedy strategy to obtain the solution with a gradually increasing pace parameter while where to optimally terminate this increasing process is difficult to determine.Besides, most SPL implementations are very sensitive to initialization and short of a theoretical result to clarify where SPL converges to with pace parameter increasing.In this paper, we propose a novel multi-objective self-paced learning (MOSPL) method to address these issues.Specifically, we decompose the objective functions as two terms, including the loss and the self-paced regularizer, respectively, and treat the problem as the compromise between these two objectives.This naturally reformulates the SPL problem as a standard multi-objective issue.A multi-objective evolutionary algorithm is used to optimize the two objectives simultaneously to facilitate the rational selection of a proper pace parameter.The proposed technique is capable of ameliorating a set of solutions with respect to a range of pace parameters through finely compromising these solutions inbetween, and making them perform robustly even under bad initialization.A good solution can then be naturally achieved from these solutions by making use of some off-the-shelf tools in multi-objective optimization.Experimental results on matrix factorization and action recognition demonstrate the superiority of the proposed method against the existing issues in current SPL research.

The Near Space Communication System is a promising and burgeoning alternative solution to the modern world's increasing communication demand. The deployment of the airships is important to the performance of the Near Space Communication System, of which the coverage and speed are conflictive. In a real world application, this system is also usually faced with regular changes of the user distribution. In this paper, we present a Tide User Model which simulates those changes of the user distribution. To optimize the deployment of the airships on the Tide User Model, a memetic algorithm based on MOEA/D with particularly designed operators is proposed aiming at providing a set of solutions as good as possible for the decision maker. We carry out the experiments on which the proposed algorithm and MOEA/D with two regular operators are compared for different settings. The results show that the proposed memetic algorithm based on MOEA/D achieves satisfying results and has better performance on the Tide User Model.

Multifactorial evolutionary algorithm is used to deal with multifactorial optimization problem which simultaneously optimizes multiple tasks. In this paper, we introduce particle swarm optimization operation into the multifactorial evolutionary algorithm, and propose a hybrid algorithm for multifactorial optimization. The major aim is to utilize particle swarm optimization operation to accelerate the convergence and improve the accuracy of solutions. Experimental comparisons between the proposed hybrid algorithm and the original multi-factorial evolutionary algorithm show that the particle swarm update operators can effectively accelerate the convergence on some benchmark problems.

Social networks are effective tools for analyzing many social topics in sociology. In the past few decades, a great deal of efforts have been made to study the balance property of social networks. This paper presents a novel bi-objective model for social network structural balance, and a multiobjective discrete particle swarm optimizer is used to optimize the bi-objective model. Each single run of the algorithm can yield a set of Pareto solutions, each of which represents a certain network partition that divides a signed network into many clusters. Consequently, by simultaneously optimizing the objectives in the proposed model, one may have many choices to analyze the balance problem. Extensive experiments compared against several other models and algorithms have been done. All the experiments indicate that the proposed model is helpful for social network structural balance analytics, and that the algorithm is effective.

In this paper, we propose a Spark-based fuzzy local information C-Means (FLICM) algorithm that provides synthetic aperture radar (SAR) image change detection. With the volume and resolution of SAR images increasing, current serial clustering algorithms are not suitable to handle big data, scalable solutions are indispensable. The proposed algorithm based on Spark framework implements FLICM algorithm according to MapReduce programming model. The membership degree is obtained in parallel during Map phase where the local information is obtained by broadcasting arrays and clustering centers are obtained during Reduce phase. With a cluster composed of five nodes, experiments on real SAR images show good performance.

In this paper, we propose a novel approach for detecting multiple changes from two multi-temporal images. Despite the development of the change vector analysis (CVA) framework and its improved version the compressed CVA (C2VA) framework, it is found that they are limited when tackling the multi-change detection task for the images with one channel. Also, the intensity itself is fragile due to the existing noise, which especially influences the detection of subtle changes. Therefore, the stacked denosing autoencoder (SDAE) which serves as a fine tool for feature extraction is employed to generate a multi-dimensional feature representations. In this way, the C2VA framework can be applied to the inner robust features so that a satisfactory performance can be guaranteed. Experimental results from two datasets show its high accuracy and moderate time complexity, which demonstrates the effectiveness of the proposed SDAE-C2VA approach.

Due to the noise interference and redundancy in multispectral images, it is promising to transform the available spectral channels into a suitable feature space for relieving noise and reducing the redundancy. The booming of deep learning provides a flexible tool to learn abstract and invariant features directly from the data in their raw forms. In this letter, we propose an unsupervised change detection technique for multispectral images, in which we combine deep belief networks (DBNs) and feature change analysis to highlight changes. First, a DBN is established to capture the key information for discrimination and suppress the irrelevant variations. Second, we map bitemporal change feature into a 2-D polar domain to characterize the change information. Finally, an unsupervised clustering algorithm is adopted to distinguish the changed and unchanged pixels, and then, the changed types can be identified by classifying the changed pixels into several classes according to the directions of feature changes. The experimental results demonstrate the effectiveness and robustness of the proposed method.

High-throughput experimental screening techniques have resulted in a large number of biological network data such as protein-protein interactions (PPI) data. The analysis of these data can enhance our understanding of cellular processes. PPI network alignment is one of the comparative analysis methods for analyzing biological networks. Research on PPI networks can identify conserved subgraphs and help us to understand evolutionary trajectories across species. Some evolutionary algorithms have been proposed for coping with PPI network alignment, but most of them are limited by the lower search efficiency due to the lack of the priori knowledge. In this paper, we propose a memetic algorithm, denoted as MeAlgn, to solve the biological network alignment by optimizing an objective function which introduces topological structure and sequence similarities. MeAlign combines genetic algorithm with a local search refinement. The genetic algorithm is to find interesting alignment solution regions, and the local search is to find optimal solutions around the regions. The proposed algorithm first develops a coarse similarity score matrix for initialization and then it uses a specific neighborhood heuristic local search strategy to find an optimal alignment. In MeAlign, the information of topological structure and sequence similarities is used to guide our mapping. Experimental results demonstrate that our algorithm can achieve a better mapping than some state-of-the-art algorithms and it makes a better balance between the network topology and nodes sequence similarities.

Multiobjective evolutionary algorithms based on decomposition (MOEA/D) decompose a multiobjective optimization problem into a set of simple optimization subproblems and solve them in a collaborative manner. A replacement scheme, which assigns a new solution to a subproblem, plays a key role in balancing diversity and convergence in MOEA/D. This paper proposes a global replacement scheme which assigns a new solution to its most suitable subproblems. We demonstrate that the replacement neighborhood size is critical for population diversity and convergence, and develop an approach for adjusting this size dynamically. A steady-state algorithm and a generational one with this approach have been designed and experimentally studied. The experimental results on a number of test problems have shown that the proposed algorithms have some advantages.

State-of-the-art multiobjective evolutionary algorithms (MOEAs) treat all the decision variables as a whole to optimize performance. Inspired by the cooperative coevolution and linkage learning methods in the field of single objective optimization, it is interesting to decompose a difficult high-dimensional problem into a set of simpler and low-dimensional subproblems that are easier to solve. However, with no prior knowledge about the objective function, it is not clear how to decompose the objective function. Moreover, it is difficult to use such a decomposition method to solve multiobjective optimization problems (MOPs) because their objective functions are commonly conflicting with one another. That is to say, changing decision variables will generate incomparable solutions. This paper introduces interdependence variable analysis and control variable analysis to deal with the above two difficulties. Thereby, an MOEA based on decision variable analyses (DVAs) is proposed in this paper. Control variable analysis is used to recognize the conflicts among objective functions. More specifically, which variables affect the diversity of generated solutions and which variables play an important role in the convergence of population. Based on learned variable linkages, interdependence variable analysis decomposes decision variables into a set of low-dimensional subcomponents. The empirical studies show that DVA can improve the solution quality on most difficult MOPs. The code and supplementary material of the proposed algorithm are available athttp://web.xidian.edu.cn/fliu/paper.html.

A decomposition approach decomposes a multiobjective optimization problem into a number of scalar objective optimization subproblems. It plays a key role in decomposition-based multiobjective evolutionary algorithms. However, many widely used decomposition approaches, originally proposed for mathematical programming algorithms, may not be very suitable for evolutionary algorithms. To help decomposition-based multiobjective evolutionary algorithms balance the population diversity and convergence in an appropriate manner, this letter proposes to impose some constraints on the subproblems. Experiments have been conducted to demonstrate that our proposed constrained decomposition approach works well on most test instances. We further propose a strategy for adaptively adjusting constraints by using information collected from the search. Experimental results show that it can significantly improve the algorithm performance.

In this paper, inclusion degree of fuzzy sets is introduced to image segmentation. The image segmentation problem is novelly modeled as the minimization of the overlapping rates between the inside and outside regions, subject to a constraint on the total length of the region boundaries. Considering the similar properties of fuzzy sets and statistical image domain, we use fuzzy membership functions to represent the inside and outside regions and utilize nonparametric density estimates to estimate them. Then, the inclusion degree of fuzzy sets is adopted to formulate the overlapping rates between the inside and outside regions. We solve the inclusion-degree-based optimization problem by deriving the associated gradient flow and applying curve evolution techniques. Experimental results on both synthetic and real images confirm the effectiveness of the proposed method. Compared with the previous active contour models formulated to solve the same nonparametric statistical segmentation problem, our method performs well in efficiency and evolution time.

With the increase of multisource data available from remote sensing platforms, it is demanding to develop unsupervised techniques for change detection from multisource data. The difference in imaging mechanism makes it difficult to carry out a direct comparison between multisource data in original observation spaces. Different sensors provide different descriptions on the same truth in low-dimension observation spaces, but the same truth indicates the comparability of multisource data in some high-dimensional feature spaces. Inspired by this, we try to solve this problem by transforming multisource data into a common high-dimension feature space. In this paper, an iterative coupled dictionary learning (CDL) model is proposed for multisource image change detection. This model aims to establish a pair of coupled dictionaries, one of which is responsible for the data from one sensor, whereas the other is responsible for the data from another sensor. The atoms from these two coupled dictionaries have a one-to-one correspondence at the same location. Such a property guarantees the transferability of the reconstruction coefficients between bitemporal patch pairs and provides us a desired mechanism to bridge multisource data and highlight changes. The contributions can be summarized as follows: CDL is designed to explore the intrinsic difference of multisource data for change detection in a high-dimension feature space, and an iterative scheme for unsupervised sample selection is proposed to keep the purity of training samples and gradually optimize the current coupled dictionaries. The experimental results have demonstrated the feasibility, effectiveness, and robustness of the proposed framework.

Band selection is an important preprocessing step for hyperspectral image processing. Many valid criteria have been proposed for band selection, and these criteria model band selection as a single-objective optimization problem. In this paper, a novel multiobjective model is first built for band selection. In this model, two objective functions with a conflicting relationship are designed. One objective function is set as information entropy to represent the information contained in the selected band subsets, and the other one is set as the number of selected bands. Then, based on this model, a new unsupervised band selection method called multiobjective optimization band selection (MOBS) is proposed. In the MOBS method, these two objective functions are optimized simultaneously by a multiobjective evolutionary algorithm to find the best tradeoff solutions. The proposed method shows two unique characters. It can obtain a series of band subsets with different numbers of bands in a single run to offer more options for decision makers. Moreover, these band subsets with different numbers of bands can communicate with each other and have a coevolutionary relationship, which means that they can be optimized in a cooperative way. Since it is unsupervised, the proposed algorithm is compared with some related and recent unsupervised methods for hyperspectral image band selection to evaluate the quality of the obtained band subsets. Experimental results show that the proposed method can generate a set of band subsets with different numbers of bands in a single run and that these band subsets have a stable good performance on classification for different data sets.

Superpixels have been widely used in lots of computer vision and image processing tasks but rarely used in topographic map processing due to the complex distribution of geographic elements in this kind of images. We propose a novel superpixel-generating method based on guided watershed transform (GWT). Before GWT, the cues of geographic element distribution and boundaries between different elements need to be obtained. A linear feature extraction method based on a compound opposite Gaussian filter and a shear transform is presented to acquire the distribution information. Meanwhile, a boundary detection method, which based on the color-opponent mechanisms of the visual system, is employed to get the boundary information. Then, both linear features and boundaries are input to the final partition procedure to obtain superpixels. The experiments show that our method has the best performance in shape control, size control, and boundary adherence, among all the comparison methods, which are classic and state of the art. Furthermore, we verify the low complexity and low cost of memory in our method through experiments, which makes it possible to deal with large-scale topographic maps.

This paper presents a novel change detection approach for synthetic aperture radar images based on deep learning. The approach accomplishes the detection of the changed and unchanged areas by designing a deep neural network. The main guideline is to produce a change detection map directly from two images with the trained deep neural network. The method can omit the process of generating a difference image (DI) that shows difference degrees between multitemporal synthetic aperture radar images. Thus, it can avoid the effect of the DI on the change detection results. The learning algorithm for deep architectures includes unsupervised feature learning and supervised fine-tuning to complete classification. The unsupervised feature learning aims at learning the representation of the relationships between the two images. In addition, the supervised fine-tuning aims at learning the concepts of the changed and unchanged pixels. Experiments on real data sets and theoretical analysis indicate the advantages, feasibility, and potential of the proposed method. Moreover, based on the results achieved by various traditional algorithms, respectively, deep learning can further improve the detection performance.

AdaBoost has attracted much attention in the machine learning community because of its excellent performance in combining weak classifiers into strong classifiers. However, AdaBoost tends to overfit to the noisy data in many applications. Accordingly, improving the antinoise ability of AdaBoost plays an important role in many applications. The sensitiveness to the noisy data of AdaBoost stems from the exponential loss function, which puts unrestricted penalties to the misclassified samples with very large margins. In this paper, we propose two boosting algorithms, referred to as RBoost1 and RBoost2, which are more robust to the noisy data compared with AdaBoost. RBoost1 and RBoost2 optimize a nonconvex loss function of the classification margin. Because the penalties to the misclassified samples are restricted to an amount less than one, RBoost1 and RBoost2 do not overfocus on the samples that are always misclassified by the previous base learners. Besides the loss function, at each boosting iteration, RBoost1 and RBoost2 use numerically stable ways to compute the base learners. These two improvements contribute to the robustness of the proposed algorithms to the noisy training and testing samples. Experimental results on the synthetic Gaussian data set, the UCI data sets, and a real malware behavior data set illustrate that the proposed RBoost1 and RBoost2 algorithms perform better when the training data sets contain noisy data.

Social balance property is an eminent feature of social networks. Many creative efforts concerning social balance have been done. This paper presents an optimization idea to solve the social balance of complex social networks. A single objective optimization model integrating the network balance and the community properties is proposed towards the social balance problem. A discrete particle swarm optimization algorithm is introduced to solve the proposed optimization model. Extensive experiments on synthetic and real-world signed networks demonstrate that the proposed model does make sense and the introduced optimization algorithm is promissing for solving the social balance problem.

Deep community can be detected by removing noise nodes or edges from a network. A centrality measure, named local Fiedler vector centrality is proposed for deep community detection. Algorithms to optimize local Fiedler vector centrality are either with high computation complexity or difficult to find the optimal solution of local Fiedler vector centrality. In this paper, a novel memetic algorithm is proposed to maximize local Fiedler vector centrality for deep community detection. Experiments of our proposed memetic algorithm on four real world networks demonstrate that our algorithm can find optimal solution of local Fiedler vector centrality and is effective to discover deep communities.

Recent years have seen the arising recognition of community detection in complex networks. Artificial immune systems, owing to their inherent properties, have been thoroughly studied and well applied to practical use. In this article, one of the well‐known artificial immune system models, named clonal selection algorithm, is introduced to reveal community structures in complex networks. By introducing a novel antibody population initialization mechanism and a novel hypermutation strategy, the proposed approach could be applied to moderate‐scale network. Besides, by optimizing an objective function called modularity density, the proposed algorithm is also capable of detecting community structure at multiple resolution levels. Experiments on both synthetic and real‐world networks demonstrate the effectiveness of the proposed method.

In this letter, a statistical active contour model exploiting the local information is proposed for classifying changed and unchanged regions in the difference image. It incorporates the local information and fuzzy logic for the purpose of enhancing the changed information and of reducing the effect of speckle noise. The fuzzy length term and the fuzzy penalty term are added into the fuzzy energy function. In particular, in order to avoid deriving the complex fuzzy membership updating function, we solve the Euler-Lagrange equation to minimize the fuzzy energy function instead of calculating the fuzzy energy alterations directly. The experimental results on three synthetic aperture radar images confirm the performance of the proposed model over some existing methods.

In this letter, we proposed a novel band selection algorithm for hyperspectral images (HSIs) based on column subset selection. The main idea of the proposed algorithm comes from the column subset selection problem in numerical linear algebra. It selects a group of bands, which maximizes the volume of the selected subset of columns. Since the high dimensionality decreases the contrast between bands, we use Manhattan distance to obtain a higher selection quality. Experimental results on real HSIs show that the proposed algorithm obtains competitively good results, in terms of classification accuracy, and is robust to noisy bands.

Robustness and accuracy are the two main challenging problems in feature-based remote sensing image registration. In this letter, a novel point-matching algorithm is proposed. An improved random sample consensus (RANSAC) algorithm called fast sample consensus (FSC) is proposed. It divides the data set in RANSAC into two parts: the sample set and the consensus set. Sample set has high correct rate and consensus set has a large number of correct matches. An iterative method is put forward to increase the number of correct correspondences. A set of measures has been used to evaluate the registration result. The performance of the proposed method is validated on the evaluation of these measures and the mosaic images. FSC can get more correct matches than RANSAC in less number of iterations, iterative selection of correct matches algorithm and removal of the imprecise points algorithm effectively increase the accuracy of the result. Extensive experimental studies compared with three state-of-the-art methods prove that the proposed algorithm is robust and accurate.

Research on network structural balance has been of great concern to scholars from diverse fields. In this paper, a two-step approach is proposed for the first time to address the network structural balance problem. The proposed approach involves evolutionary multiobjective optimization, followed by model selection. In the first step, an improved version of the multiobjective discrete particle swarm optimization framework developed in our previous work is suggested. The suggested framework is then employed to implement network multiresolution clustering. In the second step, a problem-specific model selection strategy is devised to select the best Pareto solution (PS) from the Pareto front produced by the first step. The best PS is then decoded into the corresponding network community structure. Based on the discovered community structure, imbalanced edges are determined. Afterward, imbalanced edges are flipped so as to make the network structurally balanced. Extensive experiments on synthetic and real-world signed networks demonstrate the effectiveness of the proposed approach.

This paper addresses the problem of globally stable direct adaptive backstepping neural network (NN) tracking control design for a class of uncertain strict-feedback systems under the assumption that the accuracy of the ultimate tracking error is given a priori. In contrast to the classical adaptive backstepping NN control schemes, this paper analyzes the convergence of the tracking error using Barbalat's Lemma via some nonnegative functions rather than the positive-definite Lyapunov functions. Thus, the accuracy of the ultimate tracking error can be determined and adjusted accurately a priori, and the closed-loop system is guaranteed to be globally uniformly ultimately bounded. The main technical novelty is to construct three new n th-order continuously differentiable functions, which are used to design the control law, the virtual control variables, and the adaptive laws. Finally, two simulation examples are given to illustrate the effectiveness and advantages of the proposed control method.

Hierarchical deep neural networks are currently popular learning models for imitating the hierarchical architecture of human brain. Single-layer feature extractors are the bricks to build deep networks. Sparse feature learning models are popular models that can learn useful representations. But most of those models need a user-defined constant to control the sparsity of representations. In this paper, we propose a multiobjective sparse feature learning model based on the autoencoder. The parameters of the model are learnt by optimizing two objectives, reconstruction error and the sparsity of hidden units simultaneously to find a reasonable compromise between them automatically. We design a multiobjective induced learning procedure for this model based on a multiobjective evolutionary algorithm. In the experiments, we demonstrate that the learning procedure is effective, and the proposed multiobjective model can learn useful sparse features.

The near space communication system is a burgeoning communication system. This system has many advantages over the satellite and terrestrial networks. Being built on the near space airships, the deployment of the airships has a significant impact on the performance of the system. Various factors should be taken into consideration to build such a system of which some objectives relate with each other and specific areas weight objectives differently. The evolutionary multiobjective optimization can fulfill the purpose to provide a series of choices of the deployment scheme. In this paper, a model of such a system is proposed and the deployment of airships is solved using the multiobjective evolutionary algorithm based on decomposition. Cases with different numbers of airships are tested and the Pareto fronts are obtained. In order to increase the density of the Pareto front, a local search method based on the positions of the airships is proposed. The experiment shows that the local search method can effectively increase the number of Pareto solutions obtained.

With the rapid expansion of the information on the Internet, recommender systems play an important role in filtering insignificant information and recommend satisfactory items to users. Accurately predicting the preference of users is the first priority of recommendation. Diversity is also an important objective in recommendation, which is achieved by recommending items from the so-called long tail of goods. Traditional recommendation techniques lay more emphasis on accuracy and overlook diversity. Simultaneously optimizing the accuracy and diversity is a multiobjective optimization problem, in which the two objectives are contradictory. In this paper, a multiobjective evolutionary algorithm based on decomposition is proposed for recommendation, which maximizes the predicted score and the popularity of items simultaneously. This algorithm returns lots of non-dominated solutions and each solution is a trade-off between the accuracy and diversity. The experiment shows that our algorithm can provide a series of recommendation results with different precision and diversity to a user.

For the presence of speckle noise in SAR images, many change detection methods have been developed to suppress the effect of noise. However, all these methods will result in the loss of image details, and the trade-off between detail preserving and noise removing capability has become an urgent problem remaining to be settled. In this paper, we put forward an innovation for change detection in synthetic aperture radar images. It integrates evolutionary computation into fuzzy clustering process, and considers detail preserving capability and noise removing capability as two separate objectives for multiobjective optimization, and thus transforming the change detection problem into a multiobjective optimization problem (MOP). Experiments conducted on real S AR images confirm that the new approach is efficient.

This paper studies the replacement schemes in MOEA/D and proposes a new replacement named global replacement. It can improve the performance of MOEA/D. Moreover, trade-offs between convergence and diversity can be easily controlled in this replacement strategy. It also shows that different problems need different trade-offs between convergence and diversity. We test the MOEA/D with this global replacement on three sets of benchmark problems to demonstrate its effectiveness.