We introduce the Hybrid Vector-Occupancy Field (HVOF), a new implicit 3D representation for reconstructing both open and closed surfaces from sparse point clouds. Existing approaches, such as occupancy field and signed distance fields, face severe limitations. They struggle with open surfaces, while unsigned distance field and neural vector field exhibit directional instability in complex topologies and ridge regions. HVOF addresses these challenges by incorporating a smoothly decaying occupancy field around the surface, while capturing precise local geometry using truncated displacement vectors, naturally mitigating direction-field ambiguities near ridge regions. This unified design forms a robust hybrid representation that leverages both occupancy and vector fields. To fulfill it, we design a Hybrid Field variational autoencoder including a hierarchical cross-attention encoder and dual-branch decoder that jointly learn occupancy and vector fields through continuous weighting. Extensive experiments demonstrate that HVOF consistently outperforms state-of-the-art methods across ShapeNet, ABC, and MGN datasets, accurately reconstructing both open and closed surfaces while preserving fine geometric details in complex regions.

Human Novel View Synthesis (HNVS) aims to synthesize photorealistic human images from novel viewpoints given observations from known views. Despite significant advances achieved by existing methods such as NeRF, diffusion models, and 3DGS, they still face substantial challenges in achieving stable modeling from a single image. In this paper, we introduce Dual-Constraint Human Gaussian Splatting (DcSplat), a novel, simple, and efficient 3D Gaussian-based framework for single-view 3D human reconstruction. To address occlusion-induced texture missing and depth ambiguities, we introduce two key components: a Latent Multi-View Consistency Constraint Mechanism and a Geometric Constraint Module. The former employs a Latent-space Appearance Transformer (LatentFormer) to learn semantically coherent, view-consistent appearance priors via SMPL-guided pseudo-view fusion. The latter refines noisy SMPL-based depth through a U-Net-like structure conditioned on latent appearance features. These two modules are jointly optimized to generate high-quality Gaussian parameters in a unified latent space. Extensive experiments demonstrate that DcSplat outperforms existing SOTA methods in both geometry and texture quality, while achieving fast inference and lower computational cost.

In recent years, adversarial attacks in hyperspectral image (HSI) classification have garnered increasing attention. However, existing attack methods primarily manipulate individual pixel spectral to mislead deep neural networks (DNNs) into misclassification, overlooking the physical consistency of hyperspectral data. This oversight results in adversarial samples that lack physical interpretability and suffer from low attack efficiency. To alleviate these issues, this paper proposes a sparse unmixing guided adversarial attack framework (SUGAA) to efficiently generate hyperspectral adversarial samples that satisfy physical consistency. The proposed framework first employs sparse unmixing to extract the abundance matrix of HSI, introducing adversarial perturbations to the abundance matrix to generate physically consistent adversarial samples. Additionally, SUGAA leverages the compositional similarity of materials within intra-class HSI pixels to design a class-specific perturbation generation strategy, enhancing the applicability of adversarial perturbations across pixels of the same class. To further improve optimization effectiveness, SUGAA incorporates a class-specific perturbation optimization algorithm based on momentum iterative gradients to avoid local optima, ensuring stable and efficient perturbation generation. Experimental results on real HSI datasets demonstrate that SUGAA not only generates adversarial samples with high attack performance and physical consistency but also exhibits robustness to common preprocessing transformations.

Large language models (LLMs) have demonstrated remarkable capabilities across various natural language processing tasks, but they remain vulnerable to adversarial attacks and pose significant security concerns. Existing attack methods often treat adversarial prompts as flat sequences, neglecting the rich hierarchical structure of natural language, which could limit their effectiveness. Advancing the methodologies for adversarial attacks is crucial for rigorously assessing the security of LLMs and identifying subtle vulnerabilities. This paper introduces AdvGP, a novel framework that leverages genetic programming (GP) to generate adversarial prompts for LLMs. AdvGP exploits the inherent structural similarities between GP trees and natural language syntax to optimize the structure of harmful prompts. The framework incorporates a multi-granularity hierarchical attack strategy, specialized genetic operators that leverage an assisting LLM for depth-aware crossover and multi-level mutation, and a comprehensive fitness function integrating semantic consistency and attack effectiveness. The proposed method achieves competitive attack performance on multiple LLMs, consistently generating harmful outputs despite higher perplexity than some baselines. Ablation studies confirm the significant contributions of both LLM-aided and depth-aware mechanisms to AdvGP’s effectiveness. Furthermore, transferability analysis reveals that the generated prompts are able to bypass the defenses of various state-of-the-art LLMs, such as ChatGPT and Gemini.

For expensive multiobjective optimization problems, there exists useful knowledge, e.g., the trained surrogate models, can be transferred to assist the optimization of a target optimization problem, which is termed as multi-problem surrogates. Stacking transfer is able to combine the pretrained source surrogate models and the preliminary target model with a meta-regression algorithm to transfer knowledge from source to target. However, when large-scale source models are involved in the many-problem scenarios, the less correlated sources may hurt the target performance, which is known as negative transfer. In this paper, sparse representation of the coefficients of meta-regression is considered to automatically select the most relevant source models for largely avoiding negative transfer. In the proposed many-problem surrogates, the coefficients of the source and target models are assumed to be sparse under the non-negativity and sum-to-one constraints. Then a sparse transfer stacking model is established with l1-norm of the coefficients. Next, the alternating direction method of multipliers is employed to solve the resulting constrained optimization problem by converting it into several much simpler problems. Most of the previous works assume that the costs for evaluation have no much difference and this assumption rarely holds in the real-world applications. In order to further reduce the total costs, an improved surrogate model with a cost-sensitive measure is designed to estimate the cost and select new solutions for real evaluation based on their estimated fitness, uncertainty and cost. Experimental results on synthetic and practical problems have demonstrated the superiority of the proposed many-problem surrogates.

Data-driven evolutionary algorithms (DDEAs) have achieved significant success in numerous real-world optimization problems, where exact objective functions and constraint functions do not exist, and they mainly rely on available data. However, the existing DDEAs primarily focus on improving performance through data and surrogate, without considering that the users may lack the specialized domain knowledge and sufficient computing resources required for DDEAs. To address the aforementioned issues, this paper proposes a novel paradigm called Evolutionary Learning and Optimization as a Service (ELOaaS) and investigates the potential collusion attacks between machine learning modules and evolutionary computing modules on cloud server, which may lead to privacy leakage. Consequently, a privacy-enhanced DDEA (PEDDEA) is proposed as an instantiation algorithm of ELOaaS, which is designed to tackle offline data-driven evolutionary optimization within the ELOaaS paradigm. In the proposed PEDDEA, a subspace learning-based privacy protection strategy is designed to defense the collusion attacks. Additionally, a model management strategy based on Kendall tau metric is introduced to construct high-quality surrogate ensembles. PEDDEA enables users to outsource private offline data to cloud servers, thereby approaching the optimal solution while ensuring privacy protection. Comprehensive experiments are conducted on benchmark problems and safety evaluation problems of autonomous vehicles. According to the experimental results, the proposed algorithm has significant performance advantages over existing offline DDEAs while ensuring privacy protection.

3D Change Detection (3DCD) has gradually become another research hotspot after image change detection. Recent works focus on using artificial labels for supervised or weakly-supervised training of siamese networks to segment changed points. However, labeling every points of multi-temporal point clouds is very expensive and time-consuming. In addition, these works lack effective self-supervised signals, and existing self-supervised signals often fail to capture sufficiently rich change information. To solve this problem, we assume that the powerful representation of 3D objects should model the consistency information of unchanged regions and distinguish different objects. Based on this assumption, we propose a new unsupervised framework called MUCD to learn change information of multi-temporal point clouds through bidirectional optimization of change segmentor and feature extractor. The training of network is divided into two stages. We first design a foreknowledge point contrastive loss based on the characteristics of the 3DCD task to initialize the feature extractor, and then propose a masked consistency loss to further learn the shared geometric information of unchanged regions in the multi-temporal point clouds, utilizing it as a free and powerful supervised signal to train a change segmentor. In the inference stage, only the segmentor is used to take multi-temporal point clouds as input and produce change segmentation result. Extensive experiments are conducted on SLPCCD and Urb3DCD, two real-world datasets of streets and urban buildings, to verify that our proposed unsupervised method is highly competitive and even outperforms supervised methods in scenes where semantic information changes occur, exhibiting better performance in generalization ability and robustness.

When the current physical adversarial patches cannot deceive thermal infrared detectors, the existing techniques implement adversarial attacks from scratch, such as digital patch generation, material production, and physical deployment. Besides, it is difficult to finely regulate infrared radiation. To address these issues, this paper designs an adversarial thermal display (AdvDisplay ) by assembling thermoelectric coolers (TECs) as an array. Specifically, to reduce the gap between patches in the physical and digital worlds and decrease the power of AdvDisplay device, heat transfer loss and electric power loss are designed to guide the patch optimization. In addition, a precise temperature control scheme for AdvDisplay is proposed based on proportional-integral-derivative (PID) control. Due to the accurate temperature regulation and the reusability of AdvDisplay , our method is able to improve the attack success rate and the efficiency of physical deployments. Extensive experimental results indicate that the proposed method possesses superior adversarial effectiveness compared to other methods and demonstrates strong robustness in physical attacks.

Learning representations from numerous 2D image data has shown promising performance, yet very few works apply this representations to point cloud registration. In this paper, we explore how to leverage the 2D information to assist the point cloud registration, and propose IAPReg, an Image-Assisted Partial 3D point cloud Registration framework with the multi-view images generated by the input point cloud. It is expected to enrich 3D information with 2D knowledge, and leverage 2D knowledge to assist with point cloud registration. Specifically, we create multi-view depth maps by projecting the input point cloud from several specific views, and then extract 2D and 3D features using some well-established models. To fuse the information learned from 2D and 3D modalities, inter-modality multi-view learning module is proposed to enhance geometric information and complement semantic information. Weighted SVD is a common method to reduce the impact of inaccurate correspondences on registration. However, determining the correspondence weights is not trivial. Therefore, we design a 2D-weighted SVD method, where the 2D knowledge is employed to provide weight information of correspondences. Extensive experiments perform that our method outperform the state-of-the-art method without additional 2D training data.

Robotic manipulation in real-world settings remains challenging, especially regarding robust generalization. Existing simulation platforms lack sufficient support for exploring how policies adapt to varied instructions and scenarios. Thus, they lag behind the growing interest in instruction-following foundation models like LLMs, whose adaptability is crucial yet remains underexplored in fair comparisons. To bridge this gap, we introduce GenManip, a realistic tabletop simulation platform tailored for policy generalization studies. It features an automatic pipeline via LLM-driven task-oriented scene graph to synthesize large-scale, diverse tasks using 10K annotated 3D object assets. To systematically assess generalization, we present GenManip-Bench, a benchmark of 200 scenarios refined via human-in-the-loop corrections. We evaluate two policy types: (1) modular manipulation systems integrating foundation models for perception, reasoning, and planning, and (2) end-to-end policies trained through scalable data collection. Results show that while data scaling benefits end-to-end methods, modular systems enhanced with foundation models generalize more effectively across diverse scenarios. We anticipate this platform to facilitate critical insights for advancing policy generalization in realistic conditions. All code will be available at project page.

Recent advancements in robotic manipulation have high- lighted the potential of intermediate representations for improving policy generalization. In this work, we explore grounding masks as an effective intermediate representation, balancing two key advantages: (1) effective spatial guidance that specifies target objects and placement areas while also conveying information about object shape and size, and (2) broad generalization potential driven by large-scale vision-language models pretrained on diverse grounding datasets. We introduce ROBOGROUND, a grounding-aware robotic manipulation policy that leverages grounding masks as an intermediate representation to guide policy networks in object manipulation tasks. To further explore and enhance generalization, we propose an automated pipeline for generating large-scale, simulated data with a diverse set of objects and instructions. Extensive experiments show the value of our dataset and the effectiveness of grounding masks as intermediate guidance, significantly enhancing the generalization abilities of robot policies. Code and data will be available at robo-ground.github.io.

Point cloud registration is a fundamental task in 3D computer vision. Recent advances have shown that graph-based methods are effective for outlier rejection in this context. However, existing clique-based methods impose overly strict constraints and are NP-hard, making it difficult to achieve both robustness and efficiency. While the k-core reduces computational complexity, which only considers node degree and ignores higher-order topological structures such as triangles, limiting its effectiveness in complex scenarios. To overcome these limitations, we introduce the $k$-truss from graph theory into point cloud registration, leveraging triangle support as a constraint for inlier selection. We further propose a consensus voting-based low-scale sampling strategy to efficiently extract the structural skeleton of the point cloud prior to $k$-truss decomposition. Additionally, we design a spatial distribution score that balances coverage and uniformity of inliers, preventing selections that concentrate on sparse local clusters. Extensive experiments on KITTI, 3DMatch, and 3DLoMatch demonstrate that our method consistently outperforms both traditional and learning-based approaches in various indoor and outdoor scenarios, achieving state-of-the-art results.

Federal learning provides a secure platform for sharing sensitive data, yet imposes stringent requirements on the data. Non-IID data often cannot fully enjoy the convenience it offers. When clients possess divergent feature sets, retaining only the common features is a prevalent yet suboptimal practice. This paper proposes a novel omnidirectional federated learning framework that employs a Star collaboration network designed to leverage independent information from client nodes for feature reconstruction of other clients. It establishes an approximate distribution network, reinforcing feature correlations while overcoming data isolation seen in traditional federal learning. Additionally, homomorphic encryption is utilized to ensure data security throughout the transmission process. Experimental evaluations on structured datasets demonstrate that the reconstructed prediction results closely approximate those under the condition of complete data, confirming the effectiveness of the Star network in data completion and multi-party prediction scenarios.

Existing 3D mask learning methods encounter performance bottlenecks under limited data, and our objective is to overcome this limitation. In this paper, we introduce a triple point masking scheme, named TPM, which serves as a scalable plug-and-play framework for MAE pre-training to achieve multi-mask learning for 3D point clouds. Specifically, we augment the baseline methods with two additional mask choices (i.e., medium mask and low mask) as our core insight is that the recovery process of an object can manifest in diverse ways. Previous high-masking schemes focus on capturing the global representation information but lack fine-grained recovery capabilities, so that the generated pre-training weights tend to play a limited role in the fine-tuning process. With the support of the proposed TPM, current methods can exhibit more flexible and accurate completion capabilities, enabling the potential autoencoder in the pre-training stage to consider multiple representations of a single 3D point cloud object. In addition, during the fine-tuning stage, an SVM-guided weight selection module is proposed to fill the encoder parameters for downstream networks with the optimal weight, maximizing linear accuracy and facilitating the acquisition of intricate representations for new objects. Extensive experimental results and theoretical analysis show that five baselines equipped with the proposed TPM achieve comprehensive performance improvements on various downstream tasks. Our code and models are available athttps://github.com/liujia99/TPM.

Research on adversarial attacks in remote sensing tasks have predominantly focused on designing perturbations or patches, presenting challenges in balancing attack success rate and adversarial stealthiness. Instead of focusing on designing adversarial examples under adversarial perturbation constraints to ensure stealthiness, this paper proposes a Style Transfer Enabled Adversarial Attack with Attention Mechanism (STEAM), which leverages style transfer to generate adversarial examples with high visual fidelity. Specifically, STEAM transfers distinctive styles from critical regions of source samples to attackable areas in target samples, effectively incorporating natural textures from source samples. To further refine this process, an attention mechanism is introduced to selectively extract style features from key regions of the source samples, mitigating redundancy from global style information. Additionally, selective style transfer process also includes the consideration of semantic features across different regions in target samples, ensuring a more effective attack area selection. As a result, STEAM achieves high attack success rate by utilizing proper style selected from groups of style samples, and preserving high visual fidelity through selectively transfer the natural style feature into specific attackable region in target samples. Experimental results on the UCM and WHU-RS19 datasets demonstrate that STEAM not only enhances the visual fidelity of adversarial examples but also improves the attack success rate. Furthermore, experiments against state-of-the-art adversarial defense methods highlight the adversarial attack effectiveness and robostness of STEAM compared to other adversarial attack methods.

Anomaly detection in surveillance videos aims to differentiate anomalies from regular events by discriminative representations, which has gathered considerable attention due to its significant effect to public security. However, most existing works are limited in the lack of annotated samples, and lots of approaches find it challenging to avoid the well-reconstruction of anomalous data. To alleviate these issues, we propose a dual distillation fusion framework for weakly supervised anomaly detection. We reformulate the anomaly detection problem into two steps, namely filtering anomalies and inpainting normal patterns. Each step corresponds to one branch of the dual distillation. Specifically, the dual distillation comprises the contrastive distillation module and the inpainting distillation module. The contrastive distillation optimizes the encoder to filter out abnormal features and capture key normal features, while the inpainting distillation refines the decoder to inpaint normal patterns on the encoded features. The contrastive distillation module and the inpainting distillation module are optimized iteratively in a self-training manner with video-level labeled data. Moreover, a joint optimization module is devised to effectively fuse the distilled encoder and decoder, thereby collectively improving the anomaly detection performance. During the training phase, we take into account the diversity of normal samples by selecting pseudo normal and abnormal samples with high confidence from abnormal videos. These selected samples, along with original normal frames, are then fed into the subsequent training iterations to enhance the distinguishing ability of the model. Experimental results show that our proposed method performs competitively on five benchmark datasets.

Transfer evolutionary multiobjective optimization leverages the relevant knowledge from other source problems (distinct but possibly related) to assist the optimization of the target problem of interest. Multi-problem surrogates stack multiple source surrogates to reduce the number of function evaluations of the target expensive problem. The current multi-problem surrogates only considers several source problems and the source and target problems are assumed to be homogeneous. In order to address the above issues, this paper proposes fast heterogeneous multi-problem surrogates for transfer evolutionary multiobjective optimization with a large number of surrogates. First, an iterative surrogate selection strategy is designed to select the highly relevant surrogates from the large-scale surrogate pool to avoid negative transfer. Second, heterogeneous multi-problem surrogates are established to align the features of the source and target models. Finally, an adaptive k-fold cross-validation method is proposed to obtain the predicted values of the target model with low computational costs. Experiments on the multiobjective optimization benchmark problems and multiobjective neural architecture search problems have demonstrated that the proposed method is able to avoid negative transfer in the large-scale scenarios and reduce the computational costs.

3D point cloud registration is a process of solving the geometric transformation between two point clouds. This process is an important issue in computer vision and pattern recognition. The registration methods based on geometric features are highly sensitive to the scale of feature extraction. Changes in scale can introduce inaccuracies in feature descriptions, thereby compromising the reliability of the registration results. To mitigate the impact of feature scale on the outcomes and the high-dimensional issue arising from features of different scales, we propose a method for multi-scale point cloud feature selection. We solve the high-dimensional problem of feature selection by designing a multi-task framework. By designing a mutual information dimensionality reduction method, we decomposed the high-dimensional feature selection task of different descriptors with multi-scale features into multiple related low-dimensional feature selection tasks. Then, by means of the knowledge transfer among these low-dimensional feature selection tasks, we sought the best feature subset to obtain more robust feature information. We evaluate the effectiveness of our method by conducting extensive experiments on various datasets. The experimental results show that the method outperforms other feature descriptors in terms of descriptive power and robustness and improves the effectiveness of point cloud registration.

Point cloud registration (PCR) is an important task for other point cloud tasks. Feature-based methods are widely adopted for their speed and efficiency in PCR. The descriptive capability of features extracted by a single geometric descriptor is limited. Descriptive capabilities can be improved by concatenating features extracted from multiple descriptors. However, due to the existence of redundant and irrelevant features, the correct corresponding points are difficult to match, which further affects the registration effect. We propose an evolutionary multitasking point cloud descriptor optimization method. Integrate existing descriptors to optimize descriptors with stronger description ability. Labeling features to calculate the feature importance for the registration and generating multitasks. In optimized processing, approximate evaluation which is calculated by prior correspondence saved in the database replaces the expensive searching correspondences process in the entire point cloud. Finally, a multiscale filter is developed to remove error correspondences by the geometric information from multiple scale descriptor features. Experimental demonstrate that the proposed approach can optimize a feature subset with higher-descriptive capability compared to other methods and show superior PCR performance on 14 point cloud models. This is the first paper on point cloud descriptor optimization, which provides a new idea for PCR research.

For adversarial attacks on aerial image object detection, some physical background attack methods have been proposed and demonstrated excellent performance. However, most of the methods restrict the protected object to a fixed area, and changes in the position and size of the object can affect the effectiveness of the attack. In addition, the size of the background is only related to the size of the object, without considering any reduction in the background area. To alleviate these issues, this paper proposes a novel adversarial background patch generation strategy, which generates an adversarial background patch where aircrafts parked at any position within it remain undetected. Besides we also aim to make the size of the background as small as possible to improve the concealment of the attack and reduce the economic cost. Specifically, a series of adversarial images are generated by placing the aircraft at random angles and positions on the adversarial background patch. Then, a novel background patch optimization strategy is proposed, which enhances the occlusion robustness of the adversarial background patch by weighting the adversarial loss of each image in the batch based on adversarial difficulty. In addition, this paper proposes a novel area loss for achieving optimal area size, which is used to reduce the cost of producing the patch. Finally, a multi-objective optimization method based on Pareto efficiency is introduced for balancing two conflicting losses, the adversarial loss and the area loss. Experimental results show that the adversarial background patches generated by this method have excellent attack effects in both digital and physical attacks, exhibiting strong occlusion robustness. In addition of that, the adversarial background patches generated by this method achieve effective attacks in a smaller area and reduce the physical implementation cost.

Hyperspectral imaging has demonstrated substantial advantages in enhancing classification performance in remote sensing applications due to its abundant spectral information. To address the challenges of label noise and class imbalance in hyperspectral image (HSI) classification, we propose an end-to-end Feature-Guided Network (FGN) for HSI. Instead of merely combining spatial and channel attention, FGN leverages feature-level attention interactions to enhance contextual understanding, leading to better feature extraction, especially for underrepresented classes. Furthermore, a bidirectional loss for curriculum learning (CL) is proposed to rank the HSI training data in a descending or ascending order. The top and bottom loss regularizers are designed to make the proposed model suitable for noisy and imbalanced HSI data distributions. In the phase of selecting pace parameter, a stacking ensemble curriculum learning (SECL) model is established to avoid that the outliers and noisy HSI data are involved into the CL training process. A novel instruction matrix based on sample weights is designed for base classifiers. The outputs of the base models, combined with the expected labels, form the input-output pairs for training the second-level classifier. Experiments conducted on multiple hyperspectral imbalanced datasets with noisy labels demonstrate the superior performance of our method.

The empirical studies of most existing graph neural networks (GNNs) broadly take the original node feature and adjacency relationship as single-channel input, ignoring the rich information of multiple graph channels. To circumvent this issue, the multichannel graph analysis framework has been developed to fuse graph information across channels. How to model and integrate shared (i.e., consistency) and channel-specific (i.e., complementarity) information is a key issue in multichannel graph analysis. In this article, we propose a cross-channel graph information bottleneck (CCGIB) principle to maximize the agreement for common representations and the disagreement for channel-specific representations. Under this principle, we formulate the consistency and complementarity information bottleneck (IB) objectives. To enable optimization, a viable approach involves deriving variational lower bound and variational upper bound (VarUB) of mutual information terms, subsequently focusing on optimizing these variational bounds to find the approximate solutions. However, obtaining the lower bounds of cross-channel mutual information objectives proves challenging through direct utilization of variational approximation, primarily due to the independence of the distributions. To address this challenge, we leverage the inherent property of joint distributions and subsequently derive variational bounds to effectively optimize these information objectives. Extensive experiments on graph benchmark datasets demonstrate the superior effectiveness of the proposed method.

Nowadays, pruning techniques have drawn attention to convolutional neural networks (CNNs) for reducing the consumption of computation resources. In particular, the Taylor-based method simplifies the evaluation of importance for each filter as the product of the gradient and weight value of the output features, which outperforms other methods in reductions of parameters and floating point operations (FLOPs). However, the Taylor-based method sacrifices too much accuracy when the overall pruning rate is relatively large compared with other pruning algorithms. In this article, we propose a self-adaptive attention factor (SAAF) to improve the performance of the slimmed model when conventional Taylor-based pruning is utilized under higher pruning. Specifically, SAAF can be calculated by leveraging the remaining ratio of filters at the early pruning stage of the Taylor-based method, and then, some pruned filters can be recovered for improving the accuracy of the slimmed model in terms of SAAF. It means that SAAF can protect filters from being overslimmed to eliminate the degeneration of Taylor-based pruning when the pruning rate is large as well as can compress models apparently across various datasets. We test the efficiency of SAAF on VGG-16 and ResNet-50 with CIFAR-10, Tiny-ImageNet, ImageNet-1000, and remote sensing images. Our method outperforms the traditional Taylor-based method obviously in accuracy, and there are only tiny sacrifices in the reduction of parameters and FLOPs, which is better than other pruning methods.

DNN have demonstrated vulnerability to adversarial attacks in object detection tasks. While significant progress has been made in single-spectrum attacks, cross-spectral adversarial attacks remain challenging due to the complex tradeoffs between visible and infrared domains. To address this, an evolutionary multiobjective cross-spectral attack (MoXAttack) framework, for developing adversarial patches in closed-box cross-spectral scenarios is proposed. MoXAttack incorporates a multipopulation constraint-handling technique, which uses both penalty functions and feasibility rules to guide the search process. Spectrum-aware genetic operators are introduced to enhance solution diversity and feasibility. The framework automatically optimizes the smooth to cross-spectral shared patch shape using curvature energy. In addition, MoXAttack utilizes SVD for visible spectrum texture perturbations and adjustable thermal shielding material thickness for infrared spectrum control. Experiments on the LLVIP dataset demonstrate that MoXAttack achieves competitive performance across multiple object detection models. Ablation studies reveal the positive impact of improved components on attack effectiveness. The multipatch strategy improves attack success rates by at least 17%, while optimized patch shapes outperform conventional geometric shapes by at least 25% in terms of mAP drop. In the physical world test, the proposed method shows stability in different viewing angles.

This article proposes a novel evolutionary multitasking optimization (EMTO) paradigm called divide-and-conquer EMTO, which divides the original complex optimization problem into multiple simple optimization tasks and then these tasks are optimized by EMTO concurrently to formulate the resulting solution of the original problem. The main characteristics of divide-and-conquer EMTO are that the considered problem can be divided into multiple small-scale optimization tasks and the optimal solution is the combination of solutions of all tasks. In order to achieve the optimal combined fitness of all tasks, a relative improvement function and an adaptive exploration optimization strategy are designed for dynamic resource allocation across tasks. Finally, a case study on hyperspectral unmixing is investigated in the proposed divide-and-conquer EMTO framework by dividing the hyperspectral image into several homogeneous regions to formulate multiple sparse unmixing tasks. Experiments on benchmark and sparse unmixing problems demonstrate the superiority of divide-and-conquer EMTO.

3D point cloud registration is a fundamental task in computer vision, aimed at estimating a transformation to align a pair of point clouds. For point cloud registration, where the popular methods are used to build a compatibility graph. Different methods of constructing compatibility graphs can result in different information regions to be searched in the graph, due to varying constraints. Evolutionary multitask optimization has gained attention in the field of evolutionary computation, as it enables knowledge transfer among multitasks to enhance the exploration of information. Inspired by this theory, this paper proposes a method to utilize compatibility graphs as tasks through evolutionary multitasking for solving the problem of point cloud registration. We first construct two compatibility graph tasks with different tightness constraints for point cloud registration. Due to the different constraints of the two proposed tasks, the local information of the graph of interest might be biased. This bias can be effectively utilized in the evolutionary multitasking framework to enhance the ability to discover meaningful consensus relationships in the search space of the graph. Then we map the two tasks to a unified search space by designing clique selection strategies, and carrying out knowledge transfer between the two tasks, aiming to emphasize more on the local consensus information in the graphs. Lastly, the efficacy of our proposed method is further validated on multiple registration datasets.

Novel benchmarks for multiobjective neural archi-tecture search are emerging consistently. It stimulates the need of knowledge transfer techniques to facilitate the repetitive and cumbersome search process. One-shot surrogate is hereby proposed to transfer knowledge from source problems. Specif-ically, a Pareto-aware super-surrogate construction technique is proposed aiming at efficient exploitation of knowledge from benchmarks to make transfer easier. Then, during the surrogate prediction process, appropriate sub-surrogates are sampled from the super-surrogate to jointly guide the evaluation decision. The multiobjective surrogate is also redesigned as objective-wise, decision-making, and combinative ones, so that it can make transfer flexible, as well as evade from the scalarization issue in performance measurement. This knowledge transfer scheme assists the convergence of the search process using previously constructed surrogates from multiple source problems, balancing exploration and exploitation.

Point cloud registration is a hot research topic in the field of computer vision. In recent years, the registration method based on evolutionary computation has attracted more and more attention because of its robustness to initial pose and flexibility of objective function design. However, most of the current evolutionary computation-based point cloud registration methods do not take into account the multi-view problem, that is, to capture the close relationship between point clouds from different perspectives. We fully realize that if these relations are used correctly, the registration performance can be improved. Therefore, this paper proposes an evolutionary multitasking multi-view point cloud registration method, which solves the problem of multi-view error accumulation. To ensure the unity of global and local, a two-level knowledge transfer strategy is proposed, which divides the multi-view cloud registration task into two levels. This strategy unifies the search space of two registration tasks, solves the negative transfer phenomenon, and avoids the problem of falling into the local optimum. Finally, the effectiveness of the method is verified by sufficient experiments. This method has strong robustness to noise and outliers, and can be effectively implemented in various registration scenarios.

Multi-instance point cloud registration is the problem of estimating multiple rigid transformations between two point clouds. Existing solutions rely on global spatial consistency of ambiguity and the time-consuming clustering of highdimensional correspondence features, making it difficult to handle registration scenarios where multiple instances overlap. To address these problems, we propose a maximal clique based multiinstance point cloud registration framework called PointMC. The key idea is to search for maximal cliques on the correspondence compatibility graph to estimate multiple transformations, and cluster these transformations into clusters corresponding to different instances to efficiently and accurately estimate all poses. PointMC leverages a correspondence embedding module that relies on local spatial consistency to effectively eliminate outliers, and the extracted discriminative features empower the network to circumvent missed pose detection in scenarios involving multiple overlapping instances. We conduct comprehensive experiments on both synthetic and real-world datasets, and the results show that the proposed PointMC yields remarkable performance improvements.

State-of-the-art (SOTA) adversarial attacks expose vulnerabilities in object detectors, often resulting in erroneous predictions. However, existing adversarial attacks neglect the stealth and flexibility of adversarial examples, which are crucial for conducting contextually consistent and inconspicuous attacks. To address these issues, leveraging the observed phenomenon of predicted box offsets in real-world object detection scenarios, this paper presents a novel adversarial attack framework called ShiftAttack. It leverages the concept of dense detection in prevalent object detectors, by boosting the confidence of low Intersection over Union (IoU) predictions within the positive samples (the set of predicted boxes responsible for localizing the same target), which leads to the erroneous exclusion of true positive predictions during the post-processing stage. Such a paradigm is highly stealthy as the shifted predictions seem like natural detector mistakes rather than obvious manipulations. To enhance the flexibility of ShiftAttack this paper proposes a generative approach called ShiftAttack Generator (SAG), which can not only shift predicted boxes for any target in arbitrary directions and distances but also facilitate adaptive feature exchange between pre- and post-shift regions to optimize the attack. Additionally, the proposed SAG incorporates the Dynamic Hinge Loss (DHL) to ensure the imperceptibility of perturbations, effectively mitigating the Patch-Pattern associated with the use of$\mathcal {L}_{2}$norm. Extensive experiments confirm that SAG surpasses other SOTA adversarial attacks in effectiveness, speed and stealthiness.

Evolutionary multitasking optimization (EMTO) is a new optimization paradigm proposed in the field of evolutionary computation in recent years. EMTO can solve several different optimization tasks simultaneously and facilitate superior convergence characteristics by transferring effective knowledge among the tasks. However, existing EMTO usually focuses only on facilitating convergence characteristics while neglecting the potential privacy leakage problem in the knowledge transfer between different tasks. The privacy leakage could result in considerable financial losses or severe reputation impairment, which may impede the development of EMTO in real-world applications. To solve the problem of privacy leakage in EMTO, this paper proposes a privacy-enhanced multitasking particle swarm optimization algorithm. A knowledge transfer strategy with privacy preservation is designed based on homomorphic encryption by combining multitasking particle swarm optimization. In addition, an inter-task knowledge transfer mechanism implemented in a low-dimensional subspace is introduced to reduce the extra computational burden caused by privacy preservation. Comprehensive experiments are conducted on synthetic and NAS problems to verify the effectiveness of the proposed method. According to the experimental results, the proposed method has remarkable advantages in privacy preservation compared to existing EMTO.

Block sparse signals have the characteristics of nonzero values concentrated as blocks. Therefore, block sparse recovery requires recovering the original signal from the observed signal and the measurement matrix, where the recovered error should be small and the signal satisfies the block sparsity. In this article, block sparse recovery is solved as a multiobjective problem (MOP) and the recovery error, sparsity, and the block number of the recovered signal are considered as the conflicting objectives. Furthermore, the dimensionality of real block sparse signals is often too large, which increases the difficulty of recovery. Cooperative coevolutionary (CC) can effectively alleviate the curse of dimensionality in block sparse recovery. In addition, block sparsity causes zero and nonzero variables in the signal to form natural clusters, which is similar to the decomposition and cooperation of subproblems in CC. Therefore, CC is used in the algorithm to solve the block sparse MOP and decomposes the original problem into different subproblems. Each subproblem has a defined nonzero degree that can be combined with the proposed adaptive operator to adaptively adjust the way of generating new solutions of subproblem, which encourages the complete solution to have block sparsity. For the increased number of function evaluations (FEs) caused by the decomposition, a new list-inquiry evaluation is designed to avoid repeated evaluation. Finally, the experimental results on simulated data and real data have demonstrated the effectiveness of the proposed algorithm.

Multiobjective neural architecture search (MONAS) methods based on evolutionary algorithms (EAs) are inefficient when the evaluation of each architecture incorporates parameter learning from scratch. A surrogate-assisted MONAS problem can be tough considering cold-start in surrogate construction, and the evaluation of predicted promising architectures could still be cumbersome. Previously solved MONAS problems are likely to convey useful knowledge that could assist solving the current MONAS problem. To take the benefit from knowledge of these previous practices, a framework tackling large-scale knowledge transfer is proposed. Through sparse-constraint transfer stacking, the surrogate for the current problem gets informative easily. With knee-region knowledge distillation from previously learned parameters of nondominated architectures, evaluation of current architectures could be efficient and credible. To avoid transferring knowledge from irrelevant problems, an iterative source selection algorithm is designed to avoid negative transfer. The proposed framework is analyzed under different source and target MONAS problem combinations. Results show that with the help of this framework, architectures with competitive performance could be found under limited evaluation budget.

Classification of multitemporal remote sensing (MTRS) images with only a few labels of one of these images has attracted widespread interest in recent years. It usually confronts three problems: domain increment, class increment, and class disappearance. In this article, a robust self-paced incremental learning (RSPIL) is proposed to alleviate the above problems. First, change detection is used to transfer labels from the source image to the target one for formulating a combined training set. Then, the weighted classification loss and the distillation loss are considered to ensure classification performance and minimal forgetting. In particular, a novel entropy-inhibit loss is proposed to suppress the classification capability for the disappearing classes. These losses are combined with self-paced learning (SPL) by introducing a weight variable to measure the “easiness” of the training samples, which is able to automatically acquire accurate decision boundaries from easy to hard since the combined training set generated by change detection contains noisy samples and outliers. Finally, a nearest-average-eigenvector classifier and an exemplar set management strategy based on the sample weights are designed to alleviate catastrophic forgetting (CF). Twenty-four MTRS image datasets from four areas are considered in the experiments. The classification results demonstrate that the proposed method is able to alleviate CF and achieves significant improvements on multitemporal image datasets.

Recently, the field of hyperspectral image (HSI) classification has witnessed advancements with the emergence of deep learning models. Promising approaches, such as self-supervised strategies and domain adaptation, have effectively tackled the overfitting challenges posed by limited labeled samples in HSI classification. To extract comprehensive semantic information from different types of auxiliary tasks, which view the problem from multiple perspectives, and efficiently integrate multiple tasks into a single network, this paper proposes a hybrid multi-task learning framework (HyMuT) by sharing representations across multiple tasks. Based on the similarity between the data and target classification task, we construct three auxiliary tasks that are similar, related and weakly correlated to the target task, while three corresponding multi-task learning methods are integrated. The framework establishes a backbone network with a hard parameter sharing mechanism, which handles the main task and a similar spatial mask classification task. Subsequently, a hierarchical transfer multi-task learning approach is introduced to transfer the knowledge of a spatial-spectral joint mask reconstruction task from the autoencoder to the backbone network. Furthermore, a new source domain HSI dataset is introduced as an auxiliary task weakly correlated. To solve the source domain classification task and assist the hard parameter sharing mechanism, a dual adversarial classifier based on adversarial learning is employed. This classifier effectively extracts domain and task invariance. Extensive experiments are conducted on four benchmark HSI datasets to evaluate the performance. The results demonstrate that HyMuT outperforms state-of-the-art methods. This code will be available from the website: https://github.com/HaoLiu-XDU/HyMuT.

With the rapid development of remote sensing (RS) technology, high-resolution RS image change detection (CD) has been widely used in many applications. Pixel-based CD techniques are maneuverable and widely used, but vulnerable to noise interference. Object-based CD techniques can effectively utilize the abundant spectrum, texture, shape, and spatial information but easy-to-ignore details of RS images. How to combine the advantages of pixel-based methods and object-based methods remains a challenging problem. Besides, although supervised methods have the capability to learn from data, the true labels representing changed information of RS images are often hard to obtain. To address these issues, this article proposes a novel semisupervised CD framework for high-resolution RS images, which employs small amounts of true labeled data and a lot of unlabeled data to train the CD network. A bihierarchical feature aggregation and extraction network (BFAEN) is designed to achieve the pixelwise together with objectwise feature concatenation feature representation for the comprehensive utilization of the two-level features. In order to alleviate the coarseness and insufficiency of labeled samples, a confident learning algorithm is used to eliminate noisy labels and a novel loss function is designed for training the model using true- and pseudo-labels in a semisupervised fashion. Experimental results on real datasets demonstrate the effectiveness and superiority of the proposed method.

In the field of monocular 3D detection, it is common practice to utilize scene geometric clues to enhance the detector’s performance. However, many existing works adopt these clues explicitly such as estimating a depth map and back-projecting it into 3D space. This explicit methodology induces sparsity in 3D representations due to the increased dimensionality from 2D to 3D, and leads to substantial information loss, especially for distant and occluded objects. To alleviate this issue, we propose MonoNeRD, a novel detection framework that can infer dense 3D geometry and occupancy. Specifically, we model scenes with Signed Distance Functions (SDF), facilitating the production of dense 3D representations. We treat these representations as Neural Radiance Fields (NeRF) and then employ volume rendering to recover RGB images and depth maps. To the best of our knowledge, this work is the first to introduce volume rendering for M3D, and demonstrates the potential of implicit reconstruction for image-based 3D perception. Extensive experiments conducted on the KITTI-3D benchmark and Waymo Open Dataset demonstrate the effectiveness of MonoNeRD. Codes are available at https://github.com/cskkxjk/MonoNeRD.

End-to-end deep clustering method utilizes deep neural networks to jointly learn representation features and clustering assignments. Although many k-means-friendly deep clustering models have been explored, the existing division-based methods tend to directly implement clustering with a specific number of clusters, which suffers from poor performance resulted from indistinguishable clusters, and contributes to bad local optimum. At the same time, the representation learning of fuzzy$c$-means clustering in the feature space still needs more research. In this article, a deep fuzzy curriculum clustering method with the learning strategy of clustering from easy to complex automatically is proposed to tackle the above issues. First, considering the soft flexible allocation of fuzzy$c$-means and the preservation of local structure of original data, the fuzzy clustering loss and the autoencoder's reconstruction loss are constructed to learn the embedded features and clustering centers simultaneously. Second, curriculum loss is introduced into the constraint to make clusters successively merge in line with implementing clustering from easy to complex, and realize the bottom-up deep aggregative clustering automatically. In addition, novel curriculum information is proposed as constraint to guide the merging of clusters belonging to the same class. Experimental results on four real-world datasets show the superiority of the proposal.

Classification of remote sensing image series which differ in quality and details, has impportant implications for the analysis of land cover, whereas it is expensive and time-consuming as a result of manual annotations. Fortunately, domain adaptation (DA) provides an outstanding solution to the problem. However, information loss while aligning two distributions often exists in traditional DA methods, which impacts the effect of classification with DA. To alleviate this issue, an inter-domain representation alignment and fine-tuning based network (RAFNet) is proposed for image series classification. Inter-domain representation alignment, which is fulfilled by a variational auto-encoder (VAE) trained by both source and target data, encourages reducing the discrepancy between the two marginal distributions of different domains and simultaneously preserving more data properties. As a result, RAFNet, which fuses the multi-scale aligned representations, performs classification task in the target domain after well trained with supervised learning in the source domain. Specifically, the multi-scale aligned representations of RAFNet is acquired by duplicating the frozen encoder of VAE. Then, an information based loss function is designed to fine-tune RAFNet, in which both the unchanged and changed information implied in change maps is completely used to learn the discriminative features better and make the model more generalized for the target domain. Finally, experiment studies on three datasets validate the effectiveness of RAFNet with considerable segmentation accuracy even the target data has no access to any annotated information.

In recent years, deep learning models, which possess powerful feature extraction abilities, have achieved remarkable success in the classification of hyperspectral images (HSIs). Nevertheless, a common challenge faced by most deep learning models, including few-shot learning models, is the scarcity of valid labeled samples. To address this issue, we propose a cross-domain self-taught network (CDSTN) for few-shot hyperspectral image classification. The proposed CDSTN merges domain adaptation and semi-supervised self-taught strategy to implement the few-shot learning, which utilizes adequate labeled and unlabeled samples from source as well as target domain respectively. For the feature information extraction of HSI, we propose a deep spatial-spectral feature embedded extractor composed of four residual blocks and a channel attention module. Additionally, a set of domain classifiers are introduced behind each residual block for the purpose of domain alignment by extracting more domain information at different depths of the network. Finally, plenty of unlabeled samples are assigned with pseudo labels through the trained network, and a pseudo label refinement module is designed to select the most confident pseudo label sample for each class to further enrich the labeled database of target domain. Experiments conducted on four widely used benchmark HSI data sets demonstrate that CDSTN can obtain superior and stable performance with limited labeled samples compared with some state of the arts.

Recently, maximizing mutual information has emerged as a powerful tool for unsupervised graph representation learning. Existing methods are typically effective in capturing graph information from the topology view but consistently ignore the node feature view. To circumvent this problem, we propose a novel method by exploiting mutual information maximization across feature and topology views. Specifically, we first construct the feature graph to capture the underlying structure of nodes in feature spaces by measuring the distance between pairs of nodes. Then we use a cross-view representation learning module to capture both local and global information content across feature and topology views on graphs. To model the information shared by the feature and topology spaces, we develop a common representation learning module by using mutual information maximization and reconstruction loss minimization. Here, minimizing reconstruction loss forces the model to learn the shared information of feature and topology spaces. To explicitly encourage diversity between graph representations from the same view, we also introduce a disagreement regularization to enlarge the distance between representations from the same view. Experiments on synthetic and real-world datasets demonstrate the effectiveness of integrating feature and topology views. In particular, compared with the previous supervised methods, the proposed method achieves comparable or even better performance under the unsupervised representation and linear evaluation protocol.

In this paper, we propose a new change detection method of multi-temporal synthetic aperture radar (SAR) images. Due to the ability of extracting key feature of images and robustness to noise, stacked denoising auto encoder (SDAE) has been widely used in remote sensing. However, the single SDAE stills has some limitations to handle with the speckle noise of SAR images. Therefore, we propose a new structure Bi-hierarchical SDAE for feature extraction. The first level of SDAE denoises the original image and reconstructs the difference map, and the second level extracts the superpixel-based difference features for classification. Besides, Bayesian optimization effectively improves the classification performance of feature classifier. The experimental results of the datasets in this paper show that the Bi-hierarchical SDAE and Bayesian optimization framework has high accuracy and proves its effectiveness.

In recent years, convolutional neural networks (CNNs) have shown excellent effectiveness on hyperspectral image classification (HSI) tasks. However, it is a challenge to design a suitable CNN architecture to obtain great performance according to different tasks. Different from the traditional manual design, in this paper, an evolutionary multitasking CNN architecture search framework for HSI classification is proposed to search the optimal architectures and accomplish classification of different tasks simultaneously. Through encoding the CNN architectures, the proposed algorithm is able to achieve global search in the same search space and select well-adapted individuals for evolution. In the evolutionary multitasking environment, information can be transferred between and within tasks, which can accelerate the convergence and explore good architectures through beneficial transfer. In the experiments, the effectiveness of the proposed method is demonstrated by the comparison with different methods on two common data sets.

Abstract For the existing jamming discrimination methods on the multistatic radar system, the single feature of target echo space correlation is utilised as the metric, which leads to the lack of comprehensive feature extraction and universal discrimination algorithm. In this study, a discrimination method in a multistatic radar system based on the convolutional neural network is proposed. This proposal combines the advantages of multiple‐radar systems cooperative detection technology with the convolutional neural network, and effectively applies to the field of anti‐deception jamming, which takes full advantage of unknown information of echo data to obtain multi‐dimensional, comprehensive, complete and deep feature differences besides correlation, so as to achieve a better jamming discrimination effect. The simulation results show that the proposed method can extract the multidimensional and separable essential features of echoes, and all these features have a strong degree of differentiation between targets and jamming, which effectively reduce the influence of noise and pulse number. At the same time, the influence of radar distribution on jamming discrimination under non‐ideal conditions is relieved, when the correlation coefficient of the true target reaches 0.4, the discrimination probability remains above 85%, which broadens the boundary conditions of the application process.

Landslide inventory mapping (LIM) on the basis of change detection techniques has potential significance for landslide disaster analysis. In this letter, a novel LIM approach based on the adaptive histogram-mean distance (AHMD) is proposed, which adaptively considers spatial contextual information of different landslide regions to improve the detection performance. First, to adapt the shape, size, and distribution of various landslides, an adaptive region around a pixel is extracted by a novel adaptive region extension algorithm without parameter setting. Second, the pixels within the adaptive region are taken to construct the spectral frequency histograms, and then, the adaptive histogram mean (AHM) is developed as the feature of a histogram. Third, the AHMD is defined based on the bin-to-bin (B2B) distance to measure change magnitude between the pairwise AHMs. Finally, LIM can be obtained by a supervised threshold method called double-window flexible pace search (DFPS). Experimental results tested on two real datasets with a very high spatial resolution (VHR) demonstrate the outperformance of the proposed AHMD approach with seven comparative methods.

Deep convolutional neural networks (CNNs) have achieved tremendous successes but tend to suffer from high computation costs mainly due to heavy over-parameterization, resulting in the difficulty of directly applying them to the ever-growing application demands based on low-end edge devices with strong power restriction and real-time inference requirement. Recently, there has much research attention devoted to compressing the network via pruning to address this issue. Most of the existing methods rely on some hand-designed pruning rules, which suffer from several limitations. Firstly, manually designed rules are only applicable to limited application scenarios, which can hardly generalize well in a broader scope. And these rules are typically designed based on human experience and via trial and error, and thus highly subjective. Then, channels of different layers in a network may have diverse distributions, which means the same pruning rule is not appropriate for each layer. To address these limitations, we propose a novel channel pruning scheme, in which the task-irrelevant channels are removed in a task-driven manner. Specifically, an adaptively differentiable search module is proposed to find the best pruning rule automatically for different layers in CNNs under sparsity constraints. Besides, we employed knowledge distillation to alleviate the excessive performance loss. Once the training process is finished, a compact network will be obtained by removing channels based on layer-wise pruning rules. We have evaluated the proposed method on some well-known benchmark datasets including CIFAR, MNIST, and ImageNet in comparison to several state-of-the-art pruning methods. Experimental results demonstrate the superiority of our method over the compared ones in terms of both parameters and FLOPs reduction.

Deep learning-based image inpainting has been widely studied, leading to great success. However, many methods adopt convolution and normalization operations, which will bring up some issues to affect the performance. The vanilla normalization cannot distinguish the pixels in corrupted regions from the other valid pixels, resulting in the mean and variance shifts. In addition, the limited receptive field of convolution makes it unable to capture long-range valid information directly. In order to tackle these challenges, we propose a novel deep generative model for image inpainting with two key modules, namely, the channel and spatially adaptive batch normalization (CSA-BN) module, and the selective latent-space-mapping-based contextual attention (SLSM-CA) layer. We replace the vanilla normalization with the CSA-BN module. By channel and spatially adaptive denormalization, the CSA-BN module can mitigate the spatial mean and variance shifts in each channel in a targeted way. In addition, we also integrate the SLSM-CA layer into our model to capture the long-range correlations explicitly. By introducing dual-branch attention and a feature selection module, the SLSM-CA layer can selectively utilize the multi-scale background information to improve prediction quality. What’s more, it introduces the latent spaces to achieve the low-rank approximations of attention matrices and to reduce computational costs. Extensive quantitative and qualitative evaluations demonstrate the superiority of the proposed method compared with state-of-the-art methods.

Detecting anomalies in videos is a fundamental issue in public security. The majority of existing deep learning methods often perform anomaly detection based on the behavior or the trajectory of a single target. However, due to the overlaps of the crowd and the low-resolution of monitoring images, the segmentation of population is hard to implement and the features cannot be learned thoroughly, which make the methods be easily disturbed by visual elements and thus may lead to false detection sometimes. To tackle these problems, we propose the influence-aware attention to learn the representative attributes of the whole crowd. Walking pedestrians can be divided into numbers of flows, and in this paper, we aim to measure the consistency of movement patterns in the same stream and the interactions between different streams. Meanwhile, great importance is given to the relation between pedestrians and the circumstance for certain anomalies occur as a result of environmental issues. Specifically, the influence-aware attention module is composed of the motion attention and the location attention, which is designed to quantify the relations in the scene from spatial and temporal aspects. For the lack of abnormal samples, we utilize a dual generator-based framework to learn interactions among normal scenes. Experimental results on six benchmarks verify the effectiveness and robustness of our proposed method.

Face is one of the most attractive sensitive information in visual shared data. It is an urgent task to design an effective face deidentification method to achieve a balance between facial privacy protection and data utilities when sharing data. Most of the previous methods for face deidentification rely on attribute supervision to preserve a certain kind of identity-independent utility but lose the other identity-independent data utilities. In this article, we mainly propose a novel disentangled representation learning architecture for multiple attributes preserving face deidentification called replacing and restoring variational autoencoders (R2VAEs). The R2VAEs disentangle the identity-related factors and the identity-independent factors so that the identity-related information can be obfuscated, while they do not change the identity-independent attribute information. Moreover, to improve the details of the facial region and make the deidentified face blends into the image scene seamlessly, the image inpainting network is employed to fill in the original facial region by using the deidentified face asa priori. Experimental results demonstrate that the proposed method effectively deidentifies face while maximizing the preservation of the identity-independent information, which ensures the semantic integrity and visual quality of shared images.

The bottleneck of computation burden limits the widespread use of the 2nd order optimization algorithms for training deep neural networks. In this paper, we present a computationally efficient approximation for natural gradient descent, named Swift Kronecker-Factored Approximate Curvature (SKFAC), which combines Kronecker factorization and a fast low-rank matrix inversion technique. Our research aims at both fully connected and convolutional layers. For the fully connected layers, by utilizing the low-rank property of Kronecker factors of Fisher information matrix, our method only requires inverting a small matrix to approximate the curvature with desirable accuracy. For convolutional layers, we propose a way with two strategies to save computational efforts without affecting the empirical performance by reducing across the spatial dimension or receptive fields of feature maps. Specifically, we propose two effective dimension reduction methods for this purpose: Spatial Subsampling and Reduce Sum. Experimental results of training several deep neural networks on Cifar-10 and ImageNet-1k datasets demonstrate that SKFAC can capture the main curvature and yield comparative performance to K-FAC. The proposed method bridges the wall-clock time gap between the 1st and 2nd order algorithms.

Inspired by the learning rules of humans/animals, self-paced learning (SPL) ranks the samples from easy to complex and assigns real-valued weights to the samples. The current SPL regimes adopt an increasing pace parameter to select the training samples. Obviously, it is difficult to tune the pace parameter during iterations. Furthermore, the current SPL regimes cannot go back to the previous stage even when the current model is worse than the previous one. In this article, a novel Pareto SPL (PSPL) approach is proposed to address the above issues. In PSPL, the SPL problem is considered as a multiobjective optimization problem. Then, a Pareto-based differential evolution algorithm is utilized to optimize the self-paced function and the weighted loss function simultaneously. In PSPL, a new representation method is designed to assign nonpositive weights to the unselected samples. Then, a modified mutation operator inspired by the SPL idea is proposed to generate new population based on ranking the individuals and assigning weights. No pace parameters are introduced in the proposed technique and PSPL can obtain entire solution spectrum to automatically adjust the solution path. The effectiveness of the PSPL has been demonstrated through rigorous studies with matrix factorization and multiclass classification problems.

Endmember selection of hyperspectral images is a practical yet difficult task due to the high spectral resolution and low spatial resolution of the hyperspectral cameras. The paradigm of multitask optimization has been investigated over two decades, which aim to handle multiple tasks simultaneously. To address these issues, we propose a novel multitasking framework based on multiobjective optimization evolutionary algorithm based on decomposition (MOEA/D). Specifically, we use a single population to simultaneously perform multiple subset selection tasks and apply it to a specific scene-the endmember selection of hyperspectral images. It is natural to consider that pixels in a homogeneous region of hyperspectral image as a task. Then, a within-task and between-task genetic transfer operator is constructed to reinforce the exchange of genetic material belonging to the same or different tasks for better and quicker search of the decision space. After that, this algorithm obtains a set of nondominated solutions for better decision of the active endmembers. Experiments on hyperspectral datasets show the effectiveness of our method in finding the real active endmembers.

Graph classification is a practical problem in many different domains including bioinformatics, chemoinformatics, social network analysis, and etc. For the graph classification task, the existing graph neural network approaches usually generate graph features using graph pooling at each step. However, this strategy of pooling only at the current step ignores the impact of self-loop. To eliminate this limitation, we propose a novel self-loop graph pooling strategy that can utilize the node information of the current step and the graph representation information of the previous step to generate an effective representation for the graph classification task. Further to measure the importance of self-loop, we also develop a gated approach, gated graph pooling with self-loop, that utilizes the simple fusion gate to enhance the representation capacity of the model. We evaluate our model on common benchmark datasets and experimental results have demonstrated the superior performance improvement on predictive accuracy.

Unsupervised change detection based on supervised or semisupervised classifiers has achieved strong adaptability and robustness to obtain satisfactory change detection results. However, these methods suffer from an issue that it is hard to collect reliable training samples in an unsupervised manner. In this article, a group self-paced learning (GSPL) framework is proposed to mine the reliable training samples. In the proposed method, each sample is assigned a weight to indicate its reliability. The proposed scheme is able to learn the weighted samples and update the weights iteratively in a self-paced manner to identify the reliable training samples. In the phase of updating weights, a grouping strategy is designed to avoid selecting training samples from homogeneous regions. Furthermore, a novel time-varying self-paced regularizer is proposed to automatically determine the learning scheme of self-paced learning. Finally, three classifiers, including SoftMax, backpropagation neural network, and support vector machine, are investigated under this proposed framework. Experiments on five change detection data sets demonstrate that the proposed framework can significantly outperform those state-of-art methods for change detection in terms of accuracy and robustness.

This paper investigates a new task named Conversational Question Generation (CQG) which is to generate a question based on a passage and a conversation history (i.e., previous turns of question-answer pairs).CQG is a crucial task for developing intelligent agents that can drive question-answering style conversations or test user understanding of a given passage.Towards that end, we propose a new approach named Reinforced Dynamic Reasoning (ReDR) network, which is based on the general encoder-decoder framework but incorporates a reasoning procedure in a dynamic manner to better understand what has been asked and what to ask next about the passage.To encourage producing meaningful questions, we leverage a popular question answering (QA) model to provide feedback and fine-tune the question generator using a reinforcement learning mechanism.Empirical results on the recently released CoQA dataset demonstrate the effectiveness of our method in comparison with various baselines and model variants.Moreover, to show the applicability of our method, we also apply it to create multiturn question-answering conversations for passages in SQuAD. * Work done while visiting the Ohio State University.Shelly is in second grade.She is a new student at her school.Shelly's family has lived in many different places.Shelly was born in Florida.Her family moved to Tennessee when she was two years old.When she was four years old, they moved to Texas.They moved from there to Arizona, where they now live.Q1: What grade is Shelly in ?A1: second R1: Shelly is in second grade.Q2: Was she a

Fuzzy clustering algorithm based on multiobjective optimization can achieve accurate and comprehensive clustering results. However, the estimation of objective values for this multiobjective optimization problem (MOP) might be expensive. Offspring's selection driven by simple evaluation is time consuming. Therefore, we integrate regression techniques to determine the superiority of the offspring solutions in the evolution process. However, it suffers from an issue that it is hard to collect reliable samples to train such a robust regression model. In this paper, an evolutionary multiobjective fuzzy clustering method via self-paced learning is proposed for change detection. In the proposed method, the self-paced learning process is implemented to collect reliable training samples for training a robust regression model, which can help to select promising offspring solutions from the candidate solutions for MOP. Experiments on three remote sensing image datasets demonstrate that the proposed method can significantly outperform those state-of-art methods for change detection in terms of accuracy and robustness.

Currently, the most of multitask evolutionary algorithms views multiple tasks as factors influencing the evolution of individuals. However, this consideration causes difficulty to assign fitness to individuals, because an individual which performs well on one task can have a bad performance on another task. To avoid this difficulty, this paper proposes a novel multipopulation technique for multitask optimization (MPMTO). The novelty of MPMTO is that it can solve the multiple tasks via a simple and straightforward method by corresponding each population to a task. By this way, the fitness assignment issue can be addressed by just assigning the objective value of the corresponding task to individuals. MPMTO is a general technique so that existing population-based optimization algorithms can be used in each population. This paper uses differential evolutionary algorithm in each population and develops a multipopulation multitask differential evolutionary optimization (mMTDE) based on the proposed multipopulation technique. mMTDE features that each population can use the other populations as the additional knowledge source to create an overlapping population, allowing the populations share information. By this way, the population can improve the efficacy and accuracy of solving multiple tasks. Moreover, the successful inter-task offspring can immigrate back to the corresponding population to fully utilize the inter-task knowledge. We have compared the proposed method with other state-of-the-art methods on benchmark multitask problems. The experimental results show the superiority of the proposed method which could utilizes efficiently the searching knowledge of multiple tasks.

Non-negative matrix factorization (NMF) is becoming increasingly popular in many research fields due to its particular properties of semantic interpretability and part-based representation. Sparseness constraints are usually imposed on the NMF problems in order to achieve potential features and sparse representation. These constrained NMF problems are usually reformulated as regularization models to solve conveniently. However, the regularization parameters in the regularization model are difficult to tune and the frequently used sparse-inducing terms in the regularization model generally have bias effects on the induced matrix and need an extra restricted isometry property (RIP). This paper proposes a multiobjective sparse NMF paradigm which refrains from the regularization parameter issues, bias effects, and the RIP condition. A novel multiobjective memetic algorithm is also proposed to generate a set of solutions with diverse sparsity and high factorization accuracy. A masked projected gradient local search scheme is specially designed to accelerate the convergence rate. In addition, a priori knowledge is also integrated in the algorithm to reduce the computational time in discovering our interested region in the objective space. The experimental results show that the proposed paradigm has better performance than some regularization algorithms in producing solutions with different degrees of sparsity as well as high factorization accuracy, which are favorable for making the final decisions.

Self-paced learning (SPL) is a recently proposed paradigm to imitate the learning process of humans/animals. SPL involves easier samples into training at first and then gradually takes more complex ones into consideration. Current SPL regimes incorporate a self-paced (SP) regularizer into the learning objective with a gradually increasing pace parameter. Therefore, it is difficult to obtain the solution path of the SPL regime and determine where to optimally stop this increasing process. In this paper, a multiobjective SPL method is proposed to optimize the loss function and the SP regularizer simultaneously. A decomposition-based multiobjective particle swarm optimization algorithm is used to simultaneously optimize the two objectives for obtaining the solutions. In the proposed method, a polynomial soft weighting regularizer is proposed to penalize the loss. Theoretical studies are conducted to show that the previous regularizers are roughly particular cases of the proposed polynomial soft weighting regularizer family. Then an implicit decomposition method is proposed to search the solutions with respect to the sample number involved into training. A set of solutions can be obtained by the proposed method and naturally constitute the solution path of the SPL regime. Then a satisfactory solution can be naturally obtained from these solutions by utilizing some effective tools in evolutionary multiobjective optimization. Experiments on matrix factorization and classification problems demonstrate the effectiveness of the proposed technique.

Evolutionary multitasking is a recently proposed paradigm to simultaneously solve multiple tasks using a single population. Most of the existing evolutionary multitasking algorithms treat all tasks equally and then assign the same amount of resources to each task. However, when the resources are limited, it is difficult for some tasks to converge to acceptable solutions. This paper aims at investigating the resource allocation in the multitasking environment to efficiently utilize the restrictive resources. In this paper, we design a novel multitask evolutionary algorithm with an online dynamic resource allocation strategy. Specifically, the proposed dynamic resource allocation strategy allocates resources to each task adaptively according to the requirements of tasks. We also design an adaptive method to control the resources invested into cross-domain searching. The proposed algorithm is able to allocate the computational resources dynamically according to the computational complexities of tasks. The experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-art algorithms on benchmark problems of multitask optimization.

Real-world applications typically have multiple sparse reconstruction tasks to be optimized. In order to exploit the similar sparsity pattern between different tasks, this paper establishes an evolutionary multitasking framework to simultaneously optimize multiple sparse reconstruction tasks using a single population. In the proposed method, the evolutionary algorithm aims to search the locations of nonzero components or rows instead of searching sparse vector or matrix directly. Then the within-task and between-task genetic transfer operators are employed to reinforce the exchange of genetic material belonging to the same or different tasks. The proposed method can solve multiple measurement vector problems efficiently because the length of decision vector is independent of the number of measurement vectors. Finally, a case study on hyperspectral image unmixing is investigated in an evolutionary multitasking setting. It is natural to consider a sparse unmixing problem in a homogeneous region as a task. Experiments on signal reconstruction and hyperspectral image unmixing demonstrate the effectiveness of the proposed multitasking framework for sparse reconstruction.

Machine Comprehension (MC) is one of the core problems in natural language processing, requiring both understanding of the natural language and knowledge about the world. Rapid progress has been made since the release of several benchmark datasets, and recently the state-of-the-art models even surpass human performance on the well-known SQuAD evaluation. In this paper, we transfer knowledge learned from machine comprehension to the sequence-to-sequence tasks to deepen the understanding of the text. We propose MacNet: a novel encoder-decoder supplementary architecture to the widely used attention-based sequence-to-sequence models. Experiments on neural machine translation (NMT) and abstractive text summarization show that our proposed framework can significantly improve the performance of the baseline models, and our method for the abstractive text summarization achieves the state-of-the-art results on the Gigaword dataset.

With the sparse unmixing becoming increasingly popular recently, some advanced regularization algorithms have been proposed for settling this problem. However, they are limited by their “decision ahead of solution” attribute, i.e., the regularization parameters must be preset before the solution is obtained. In this paper, the sparse unmixing problem is first formulated as a two-phase multiobjective problem. The first phase simultaneously minimizes the unmixing residuals and the number of estimated endmembers for automatically finding the real active endmembers from the spectral library. A decomposition-based endmember selection algorithm considering the gene exchange in the population is specially designed for better and quicker search of the decision space. This algorithm can obtain a set of nondominated solutions for better decision of the active endmembers, which are important for the subsequent calculation of the abundance matrix. The second phase concurrently minimizes the unmixing residuals and the total variation term for estimating a preferable abundance matrix. A local search strategy based on the multiplicative update rule is designed in the evolution process for better approximation of the Pareto front. The experimental results on the synthetic as well as the real data reveal that the proposed framework has a better performance in finding the real active endmembers and estimating their corresponding abundances than some advanced regularization algorithms.

This paper focuses on the connecting structure of deep neural networks and proposes a layerwise structure learning method based on multiobjective optimization. A model with better generalization can be obtained by reducing the connecting parameters in deep networks. The aim is to find the optimal structure with high representation ability and better generalization for each layer. Then, the visible data are modeled with respect to structure based on the products of experts. In order to mitigate the difficulty of estimating the denominator in PoE, the denominator is simplified and taken as another objective, i.e., the connecting sparsity. Moreover, for the consideration of the contradictory nature between the representation ability and the network connecting sparsity, the multiobjective model is established. An improved multiobjective evolutionary algorithm is used to solve this model. Two tricks are designed to decrease the computational cost according to the properties of input data. The experiments on single-layer level, hierarchical level, and application level demonstrate the effectiveness of the proposed algorithm, and the learned structures can improve the performance of deep neural networks.

Endmember extraction is a critical step of spectral unmixing. In this paper, a novel endmember extraction algorithm based on evolutionary multi-objective optimization is proposed for hyperspectral remote sensing images. In the proposed method, endmember extraction is modeled as a multi-objective optimization problem. Then the root mean square error between the original image and its remixed image and the number of endmembers are chosen as two conflicting objective functions, which are simultaneously optimized by particle swarm optimization algorithm to find the best tradeoff solutions. In order to promote diversity and speed up the convergence of the algorithm, a new particle status updating strategy and a novel method for selecting leaders are designed. The experimental results on both simulated and real hyperspectral remote sensing images confirm the performance of the proposed approach over some existing methods.

Band selection is a crucial preprocessing step for hyperspectral image classification, which is a classic feature selection method. Feature selection is designed to select feature subsets to represent the whole feature space. For feature selection, two crucial issues need to be handled: preserving information and redundancy reducing. In this paper, a novel feature selection method for hyperspectral image classification is proposed, which is based on a newly designed memetic algorithm. In the proposed method, a suitable objective function is designed, which can measure the contained crucial information and redundancy information in the selected feature subsets. To optimize this objective function efficiently, a novel memetic algorithm is designed. The genetic operator and local search strategy are newly designed according to the characteristic of hyperspectral images. Experiments are implemented on three real data sets compared with some state of arts. The experimental results show that the proposed method can obtain stable and superior feature subsets for classification.

Convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks. In order to distinguish the reliable data from the noisy and confusing data, we improve CNNs with self-paced learning (SPL) for enhancing the learning robustness of CNNs. In the proposed self-paced convolutional network (SPCN), each sample is assigned to a weight to reflect the easiness of the sample. Then a dynamic self-paced function is incorporated into the leaning objective of CNN to jointly learn the parameters of CNN and the latent weight variable. SPCN learns the samples from easy to complex and the sample weights can dynamically control the learning rates for converging to better values. To gain more insights of SPCN, theoretical studies are conducted to show that SPCN converges to a stationary solution and is robust to the noisy and confusing data. Experimental results on MNIST and rectangles datasets demonstrate that the proposed method outperforms baseline methods.

Recently, Recurrent Neural Network (RNN) solutions for recommender systems (RS) are becoming increasingly popular. The insight is that, there exist some intrinsic patterns in the sequence of users' actions, and RNN has been proved to perform excellently when modeling sequential data. In traditional tasks such as language modeling, RNN solutions usually only consider the sequential order of objects without the notion of interval. However, in RS, time intervals between users' actions are of significant importance in capturing the relations of users' actions and the traditional RNN architectures are not good at modeling them. In this paper, we propose a new LSTM variant, i.e. Time-LSTM, to model users' sequential actions. Time-LSTM equips LSTM with time gates to model time intervals. These time gates are specifically designed, so that compared to the traditional RNN solutions, Time-LSTM better captures both of users' short-term and long-term interests, so as to improve the recommendation performance. Experimental results on two real-world datasets show the superiority of the recommendation method using Time-LSTM over the traditional methods.

Sparse unmixing of hyperspectral data is an important technique aiming at estimating the fractional abundances of the end members. Traditional sparse unmixing is faced with the l0-norm problem which is an NP-hard problem. Sparse unmixing is inherently a multiobjective optimization problem. Most of the recent works combine cost functions into single one to construct an aggregate objective function, which involves weighted parameters that are sensitive to different data sets and difficult to tune. In this paper, a novel multiobjective cooperative coevolutionary algorithm is proposed to optimize the reconstruction term, the sparsity term and the total variation regularization term simultaneously. A problem-dependent cooperative coevolutionary strategy is designed because sparse unmixing encounters a large scale optimization problem. The proposed approach optimizes the nonconvex l0-norm problem directly and can find a better compromise between two or more competing cost function terms automatically. Experimental results on simulated and real hyperspectral data sets demonstrate the effectiveness of the proposed method.

Current self-paced learning (SPL) regimes adopt the greedy strategy to obtain the solution with a gradually increasing pace parameter while where to optimally terminate this increasing process is difficult to determine.Besides, most SPL implementations are very sensitive to initialization and short of a theoretical result to clarify where SPL converges to with pace parameter increasing.In this paper, we propose a novel multi-objective self-paced learning (MOSPL) method to address these issues.Specifically, we decompose the objective functions as two terms, including the loss and the self-paced regularizer, respectively, and treat the problem as the compromise between these two objectives.This naturally reformulates the SPL problem as a standard multi-objective issue.A multi-objective evolutionary algorithm is used to optimize the two objectives simultaneously to facilitate the rational selection of a proper pace parameter.The proposed technique is capable of ameliorating a set of solutions with respect to a range of pace parameters through finely compromising these solutions inbetween, and making them perform robustly even under bad initialization.A good solution can then be naturally achieved from these solutions by making use of some off-the-shelf tools in multi-objective optimization.Experimental results on matrix factorization and action recognition demonstrate the superiority of the proposed method against the existing issues in current SPL research.

In this paper, inclusion degree of fuzzy sets is introduced to image segmentation. The image segmentation problem is novelly modeled as the minimization of the overlapping rates between the inside and outside regions, subject to a constraint on the total length of the region boundaries. Considering the similar properties of fuzzy sets and statistical image domain, we use fuzzy membership functions to represent the inside and outside regions and utilize nonparametric density estimates to estimate them. Then, the inclusion degree of fuzzy sets is adopted to formulate the overlapping rates between the inside and outside regions. We solve the inclusion-degree-based optimization problem by deriving the associated gradient flow and applying curve evolution techniques. Experimental results on both synthetic and real images confirm the effectiveness of the proposed method. Compared with the previous active contour models formulated to solve the same nonparametric statistical segmentation problem, our method performs well in efficiency and evolution time.

In this letter, a statistical active contour model exploiting the local information is proposed for classifying changed and unchanged regions in the difference image. It incorporates the local information and fuzzy logic for the purpose of enhancing the changed information and of reducing the effect of speckle noise. The fuzzy length term and the fuzzy penalty term are added into the fuzzy energy function. In particular, in order to avoid deriving the complex fuzzy membership updating function, we solve the Euler-Lagrange equation to minimize the fuzzy energy function instead of calculating the fuzzy energy alterations directly. The experimental results on three synthetic aperture radar images confirm the performance of the proposed model over some existing methods.

Hierarchical deep neural networks are currently popular learning models for imitating the hierarchical architecture of human brain. Single-layer feature extractors are the bricks to build deep networks. Sparse feature learning models are popular models that can learn useful representations. But most of those models need a user-defined constant to control the sparsity of representations. In this paper, we propose a multiobjective sparse feature learning model based on the autoencoder. The parameters of the model are learnt by optimizing two objectives, reconstruction error and the sparsity of hidden units simultaneously to find a reasonable compromise between them automatically. We design a multiobjective induced learning procedure for this model based on a multiobjective evolutionary algorithm. In the experiments, we demonstrate that the learning procedure is effective, and the proposed multiobjective model can learn useful sparse features.

For the presence of speckle noise in SAR images, many change detection methods have been developed to suppress the effect of noise. However, all these methods will result in the loss of image details, and the trade-off between detail preserving and noise removing capability has become an urgent problem remaining to be settled. In this paper, we put forward an innovation for change detection in synthetic aperture radar images. It integrates evolutionary computation into fuzzy clustering process, and considers detail preserving capability and noise removing capability as two separate objectives for multiobjective optimization, and thus transforming the change detection problem into a multiobjective optimization problem (MOP). Experiments conducted on real S AR images confirm that the new approach is efficient.

Since learning action selection strategy is time-consuming due to the reinforcement learning algorithm, a heuristic reinforcement learning algorithm is presented based on the state backtracking reinforcement learning to improve the action selection strategy of the reinforcement learning. The selection strategies of repeated the action are analyzed and compared by state backtracking. A cost function is defined to denote the importance of repetitive actions. A novel heuristic function is given by combing the action-reward with the cost of an action. This algorithm reinforces the important of an action by heuristic function to speed learning and reduces unnecessary explorations by the cost function, so as to steadily improve the learning efficiency. The simulation results of two robot games proves that the algorithm can effectively enhancement the learning rate of Q-learning based on the state backtracking heuristic reinforcement learning method.