Vision foundation models (VFMs) have demonstrated remarkable capabilities in learning universal visual representations. However, adapting these models to downstream tasks conventionally requires parameter updates, with even parameter-efficient fine-tuning methods necessitating the modification of thousands to millions of weights. In this paper, we investigate the redundancies in the segment anything model (SAM) and then propose a novel parameter-free fine-tuning method. Unlike traditional fine-tuning methods that adjust parameters, our method emphasizes selecting, reusing, and enhancing pre-trained features, offering a new perspective on fine-tuning foundation models. Specifically, we introduce a channel selection algorithm based on the model's output difference to identify redundant and effective channels. By selectively replacing the redundant channels with more effective ones, we filter out less useful features and reuse more task-irrelevant features to downstream tasks, thereby enhancing the task-specific feature representation. Experiments on both out-of-domain and in-domain datasets demonstrate the efficiency and effectiveness of our method in different vision tasks (e.g., image segmentation, depth estimation and image classification). Notably, our approach can seamlessly integrate with existing fine-tuning strategies (e.g., LoRA, Adapter), further boosting the performance of already fine-tuned models. Moreover, since our channel selection involves only model inference, our method significantly reduces GPU memory overhead.

Spiking Neural Networks (SNNs) promise significant energy efficiency by processing information via sparse, event-driven spikes. However, realizing this potential is hindered by the conventional use of a rigid, uniform timestep, T. This constraint imposes a challenging trade-off between accuracy and latency, while also incurring the prohibitive training costs of Backpropagation Through Time (BPTT). To overcome this limitation, we introduce the Pseudo-Spiking Neuron (PseudoSN), a novel training proxy that conceptualizes latency as an intrinsic, learnable parameter for each neuron. Building on the efficiency of rate-based methods, the PseudoSN models temporal dynamics in a single, BPTT-free pass. It employs a learnable probabilistic noise scheme to emulate the discretization effects of spike generation (e.g., clipping and quantization), making the neuron-specific timestep—and thus latency—directly optimizable via backpropagation. Integrated into a hardware-aware objective, our framework trains heterogeneous-latency SNNs that autonomously learn to optimize the trade-offs among accuracy, latency and energy, establishing a new state-of-the-art on major benchmarks.

Knowledge distillation from Artificial Neural Networks (ANNs) to Spiking Neural Networks (SNNs) is a prominent training paradigm. However, its efficacy is fundamentally limited by a spectral mismatch: SNNs, with their intrinsic low-pass filtering characteristics, struggle to learn high-frequency details from their ANN teachers, creating a bottleneck in knowledge transfer at both the feature and logit levels. To address this, we propose Bi-Spectrum Distillation (BSD), a novel framework that mitigates the mismatch from two complementary perspectives. First, at the feature level, our Spectral Residual Distillation (SRD) enhances the student SNN's features with a parameter-efficient, learnable filter that adaptively compensates for high-frequency information loss, which transforms the student's output to better match the teacher's rich spectral target. Second, at the logits level, our Spectral Semantic Distillation (SSD) enhances fine-grained classification by distilling high-frequency components from teacher-ordered logits. Extensive experiments on CIFAR-10/100, ImageNet, and CIFAR10-DVS demonstrate that BSD achieves new state-of-the-art performance across both CNN and Transformer-based SNNs, validating its effectiveness and broad applicability.

Diffusion models (DMs) have recently demonstrated remarkable success in modeling large-scale data distributions. However, many downstream tasks require guiding the generated content based on specific differentiable metrics, typically necessitating backpropagation during the generation process. This approach is computationally expensive, as generating with DMs often demands tens to hundreds of recursive network calls, resulting in high memory usage and significant time consumption. In this paper, we propose a more efficient alternative that approaches the problem from the perspective of parallel denoising. We show that full backpropagation throughout the entire generation process is unnecessary. The downstream metrics can be optimized by retaining the computational graph of only one step during generation, thus providing a shortcut for gradient propagation. The resulting method, which we call Shortcut Diffusion Optimization (SDO), is generic, high-performance, and computationally lightweight, capable of optimizing all parameter types in diffusion sampling. We demonstrate the effectiveness of SDO on several real-world tasks, including controlling generation by optimizing latent and aligning the DMs by fine-tuning network parameters. Compared to full backpropagation, our approach reduces computational costs by $\sim\! 90\%$∼90% while maintaining superior performance. Code is available at https://github.com/deng-ai-lab/SDO.

The automatic design of soft robots is an intertwined process of evolving morphology and learning control. As reinforcement learning is repeatedly used to learn the control policy for each candidate robot design, the design process becomes time-consuming. So far, the common design paradigm in robotics has been based on a single task. In fact, there is control similarity between different tasks. Learning a controller with combinatorial generalization capabilities across a variety of tasks can significantly reduce the computational cost of the design process. To this end, we propose a cross-task collaborative evolutionary algorithm that constructs a universal controller capable of solving a group of tasks simultaneously. Instead of “one robot, one controller, one task" paradigm, the proposed universal controller is to learn a control policy, which can generalize to unseen morphologies. After the controller learning on easy tasks, the universal controller can be further transferred to new hard tasks. Furthermore, the knowledge transfer is incorporated in the search strategy to enhance the performance of the universal controller. The experimental results on 13 test tasks demonstrate that the proposed algorithm outperforms the SOTA design algorithms on 8 of them. Compared to these algorithms, the proposed algorithm reduces the computational cost by 55% while achieving comparable performance, particularly for unseen hard tasks.

The Segment Anything Model (SAM) is a widely used vision foundation model with diverse applications, including image segmentation, detection, and tracking. Given SAM's wide applications, understanding its robustness against adversarial attacks is crucial for real-world deployment. However, research on SAM's robustness is still in its early stages. Existing attacks often overlook the role of prompts in evaluating SAM's robustness, and there has been insufficient exploration of defense methods to balance the robustness and accuracy. To address these gaps, this paper proposes an adversarial robustness framework designed to evaluate and enhance the robustness of SAM. Specifically, we introduce a cross-prompt attack method to enhance the attack transferability across different prompt types. Besides attacking, we propose a few-parameter adaptation strategy to defend SAM against various adversarial attacks. To balance robustness and accuracy, we use the singular value decomposition (SVD) to constrain the space of trainable parameters, where only singular values are adaptable. Experiments demonstrate that our cross-prompt attack method outperforms previous approaches in terms of attack success rate on both SAM and SAM 2. By adapting only 512 parameters, we achieve at least a 15% improvement in mean intersection over union (mIoU) against various adversarial attacks. Compared to previous defense methods, our approach enhances the robustness of SAM while maximally maintaining its original performance.

Diffusion models, recognized for their effectiveness as generative priors, have become essential tools for addressing a wide range of visual challenges. Recently, there has been a surge of interest in leveraging Denoising processes for Regularization (DR) to solve inverse problems. However, existing methods often face issues such as mode collapse, which results in excessive smoothing and diminished diversity. In this study, we perform a comprehensive analysis to pinpoint the root causes of gradient inaccuracies inherent in DR. Drawing on insights from diffusion model distillation, we propose a novel approach called Consistency Regularization (CR), which provides stabilized gradients without the need for ODE simulations. Building on this, we introduce Hybrid Regularization (HR), a unified framework that combines the strengths of both DR and CR, harnessing their synergistic potential. Our approach proves to be effective across a broad spectrum of inverse problems, encompassing both linear and nonlinear scenarios, as well as various measurement noise statistics. Experimental evaluations on benchmark datasets, including FFHQ and ImageNet, demonstrate that our proposed framework not only achieves highly competitive results compared to state-of-the-art methods but also offers significant reductions in wall-clock time and memory consumption.

Adversarial patches are widely used to evaluate the robustness of object detection systems in real-world scenarios. These patches were initially designed to deceive single-modal detectors (e.g., visible or infrared) and have recently been extended to target visible-infrared dual-modal detectors. However, existing dual-modal adversarial patch attacks have limited attack effectiveness across diverse physical scenarios. To address this, we propose CDUPatch, a universal cross-modal patch attack against visible-infrared object detectors across scales, views, and scenarios. Specifically, we observe that color variations lead to different levels of thermal absorption, resulting in temperature differences in infrared imaging. Leveraging this property, we propose an RGB-to-infrared adapter that maps RGB patches to infrared patches, enabling unified optimization of cross-modal patches. By learning an optimal color distribution on the adversarial patch, we can manipulate its thermal response and generate an adversarial infrared texture. Additionally, we introduce a multi-scale clipping strategy and construct a new visible-infrared dataset, MSDrone, which contains aerial vehicle images in varying scales and perspectives. These data augmentation strategies enhance the robustness of our patch in real-world conditions. Experiments on four benchmark datasets (e.g., DroneVehicle, LLVIP, VisDrone, MSDrone) show that our method outperforms existing patch attacks in the digital domain. Extensive physical tests further confirm strong transferability across scales, views, and scenarios. Attack demos are provided in the supplementary materials.

Black-box adversarial attacks can be categorized into transfer-based and query-based attacks. The former usually has poor transfer performance due to the mismatch between the architectures of models, while the query-based attacks require massive queries and high dimensional optimization variables. In order to solve the above problems, we propose a novel attack framework integrating the advantages of transfer- and query-based attacks, where the framework is divided into two phases: training the adversarial generator and executing the black-box attacks. In the first stage, a generator is trained by the adversarial loss function so that it can output adversarial perturbation, where the latent variables are designed as the input of the generator to reduce the dimension of the optimization variables. In the second stage, based on the trained generator, we further employ a particle swarm optimization algorithm to optimize the latent variables so that the generator can output the perturbation that can achieve a successful attack. Extensive experiments are performed on the ImageNet dataset, and the results demonstrate that the proposed framework can obtain better attack performance compared with a number of the state-of-the-art black-box adversarial attack methods. In addition, we show the flexibility of the proposed framework by extending the experiment for few-pixel attacks.

Due to the urgent need of the robustness of deep neural networks (DNN), numerous existing open-sourced tools or platforms are developed to evaluate the robustness of DNN models by ensembling the majority of adversarial attack or defense algorithms. Unfortunately, current platforms can neither optimize the DNN architectures nor the configuration of adversarial attacks to further enhance the model robustness or the performance of adversarial attacks. To alleviate these problems, in this paper, we propose a novel platform called auto-adversarial attack and defense ($A^{3}D$A3D), which can help search for robust neural network architectures and efficient adversarial attacks. $A^{3}D$A3D integrates multiple neural architecture search methods to find robust architectures under different robustness evaluation metrics. Besides, we provide multiple optimization algorithms to search for efficient adversarial attacks. In addition, we combine auto-adversarial attack and defense together to form a unified framework. Among auto adversarial defense, the searched efficient attack can be used as the new robustness evaluation to further enhance the robustness. In auto-adversarial attack, the searched robust architectures can be utilized as the threat model to help find stronger adversarial attacks. Experiments on CIFAR10, CIFAR100, and ImageNet datasets demonstrate the feasibility and effectiveness of the proposed platform.

Object detectors have demonstrated vulnerability to adversarial examples crafted by small perturbations that can deceive the object detector. Existing adversarial attacks mainly focus on white-box attacks and are merely valid at a specific viewpoint, while the universal multi-view black-box attack is less explored, limiting their generalization in practice. In this paper, we propose a novel universal multi-view black-box attack against object detectors, which optimizes a universal adversarial UV texture constructed by multiple image stickers for a 3D object via the designed layout optimization algorithm. Specifically, we treat the placement of image stickers on the UV texture as a circle-based layout optimization problem, whose objective is to find the optimal circle layout filled with image stickers so that it can deceive the object detector under the multi-view scenario. To ensure reasonable placement of image stickers, two constraints are elaborately devised. To optimize the layout, we adopt the random search algorithm enhanced by the devised important-aware selection strategy to find the most appropriate image sticker for each circle from the image sticker pools. Extensive experiments conducted on four common object detectors suggested that the detection performance decreases by a large magnitude of 74.29% on average in multi-view scenarios. Additionally, a novel evaluation tool based on the photo-realistic simulator is designed to assess the texture-based attack fairly.

Generative models represented by diffusion models have recently shown great potential in image generation. They usually use a reverse iteration process to map noise into the data. However, for many real-world applications such as image restoration and translation, the model input comes from a distribution that is not random noise, making it difficult for these models to adapt directly to these tasks. In this paper, we introduce Image-to-Image Bayesian Flow Networks (I2I-BFNs), a novel framework for general-purpose image-to-image translation (I2I) that operates within the parameter space of distributions. This method upholds Gaussian distributions over pixel intensities, refining distribution parameters through closed-form Bayesian inference, steered by the network's predictions for the target image. An essential aspect of our approach is the utilization of the conditional image as a robust prior parameter, initializing the translation process from a deterministic, clean image to reduce variance and produce interpretable generation. Additionally, we introduce a skip sampling technique that enhances the efficiency of I2I-BFNs, facilitating rapid translation in diverse image restoration and general I2I tasks. Our experimental evaluations showcase the model's competitive edge in various settings, underscoring its efficacy and adaptability. This work contributes new insights and opportunities for the large-scale development of efficient conditional generation systems.

Neural processes (NPs) have recently emerged as a powerful meta-learning framework capable of making predictions based on an arbitrary number of context points. However, the learning of NPs and their variants is hindered by the need for explicit reliance on the log-likelihood of predictive distributions, which complicates the training process. To tackle this problem, we introduce score-based NP (SNP) models, drawing inspiration from recently developed score-based generative models (SGMs) that restore data from noise by reversing a perturbation process. With denoising score matching (DSM) techniques, the SNPs bypass the intractable log-likelihood calculations, learning parameterized score functions instead. We also demonstrate that score functions possess excellent attributes that enable us to represent a wide family of conditional distributions naturally. Moreover, as data points are inherently unordered, it is crucial to incorporate appropriate inductive biases into SNPs. To this end, we propose building blocks for parameterizing permutation equivariant score functions, which induce the SNPs with the desired properties. Through extensive experimentation on both synthetic and real-world datasets, our SNPs exhibit remarkable performance and outperform existing state-of-the-art NP approaches.

Adversarial transferability is a significant property of adversarial examples, which renders the adversarial example capable of attacking unknown models. However, the models with different architectures on the same task would concentrate on different information, which weakens adversarial transferability. To enhance the adversarial transferability, input transformation-based attacks perform random transformation over input to find a better result that can resist such transformations, but these methods ignore the model discrepancy; ensemble attacks fuse multiple models to shrink the search space to ensure that the found adversarial examples work on these models, but ensemble attacks are resource-intensive. In this article, we propose a simple but effective feature augmentation attack (FAUG) method to improve adversarial transferability. We dynamically add random noise to intermediate features of the target model during the generation of adversarial examples, thereby avoiding overfitting the target model. Specifically, we first explore the noise tolerance of the model and disclose the discrepancy under different layers and noise strengths. Then, based on that analysis, we devise a dynamic random noise generation method, which determines noise strength according to the produced features in the mini-batch. Finally, we exploit the gradient-based attack algorithm on featureaugmented models, resulting in better adversarial transferability without introducing extra computation costs. Extensive experiments conducted on the ImageNet dataset across CNN and Transformer models corroborate the efficacy of our method, e.g., we achieve improvement of +30.67% and +5.57% on input transformation-based attacks and combination methods, respectively.

Soft robot design is an intricate field with unique challenges due to its complex and vast search space. In the past literature, evolutionary computation algorithms, including novel probabilistic generative models (PGMs), have shown potential in this realm. However, these methods are sample inefficient and predominantly focus on rigid robots in locomotion tasks, which limit their performance and application in robot design automation. In this work, we propose MorphVAE, an innovative PGM that incorporates a multi-task training scheme and a meticulously crafted sampling technique termed ``continuous natural selection'', aimed at bolstering sample efficiency. This method empowers us to gain insights from assessed samples across diverse tasks and temporal evolutionary stages, while simultaneously maintaining a delicate balance between optimization efficiency and biodiversity. Through extensive experiments in various locomotion and manipulation tasks, we substantiate the efficiency of MorphVAE in generating high-performing and diverse designs, surpassing the performance of competitive baselines.

Patch-based attacks are a major black-box attack paradigm, where there is no limit to the intensity of the perturbation. The existing patch-based attack methods focus on obtaining the optimal position, shape, and pixel values against adversarial patches, however, the generated patch looks conspicuous and makes it easy to attract people's attention. Quick response(QR) code has been widely used in various fields, such as image copyright protection, stored image information. Further, it does not get noticed when a QR code is attached to the image. Therefore, we propose a deceptive texture-based black-box adversarial attack method to address the above problem. Specifically, we use the QR code pattern as the basis of the adversarial patches. Then, we model the adversarial attack as a discrete optimization problem, where the optimization variables are designed as the center coordinates of the patch pasting locations and the pixel values. Further, an upsampling technique is introduced to reduce the dimension of the optimization variables. Finally, genetic algorithm is employed as the optimizer to obtain the optimal parameter of the patch. In order to verify the effectiveness of the proposed method, we compare a number of the state-of-the-art patch-based attack methods on the ImageNet dataset, and the experimental results show that the proposed method can effectively generate deceptive adversarial examples in both digital and physical space and obtain the best attack performance, especially for the defense models.

In practical applications, combinatorial optimization problems demonstrate intrinsic complexity, predominantly characterized by discrete decision variables and fitness evaluations that are both expensive and subject to noise. Surrogate-assisted evolutionary algorithms (SAEAs) are commonly used to solve expensive optimization problems in which expensive fitness evaluations are replaced by computationally cheaper surrogate models. The quality and quantity of training data are two crucial factors affecting surrogate models' accuracy, especially in noisy environments. Implicit and explicit averaging are two straightforward and effective noise-tolerant techniques, both entirely applicable to combinatorial optimization problems. In scenarios where fitness evaluations are subject to noise, and the allotted number of evaluations is constrained, implicit averaging tends to yield a considerable quantity of training data with diminished quality, whereas explicit averaging exhibits the opposite trend. This paper discusses which of these two noise-tolerant techniques is more suitable for embedding into SAEAs. The results of six multidimensional knapsack problems show that explicit averaging is a good choice, regardless of whether the noise type is additive or multiplicative.

In the field of robotic control, designing individual controllers for each robot leads to high computational costs. Universal control policies, applicable across diverse robot morphologies, promise to mitigate this challenge. Predominantly, models based on Graph Neural Networks (GNN) and Transformers are employed, owing to their effectiveness in capturing relational dynamics across a robot’s limbs. However, these models typically employ homogeneous graph structures that overlook the functional diversity of different limbs. To bridge this gap, we introduce HeteroMorpheus, a novel method based on heterogeneous graph Transformer. This method uniquely addresses limb heterogeneity, fostering better representation of robot dynamics of various morphologies. Through extensive experiments we demonstrate the superiority of HeteroMorpheus against state-of-the-art methods in the capability of policy generalization, including zero-shot generalization and sample-efficient transfer to unfamiliar robot morphologies.

Abstract Due to the powerful ability in capturing the global information, transformer has become an alternative architecture of CNNs for hyperspectral image classification. However, general transformer mainly considers the global spectral information while ignores the multiscale spatial information of the hyperspectral image. In this paper, we propose a multiscale spectral–spatial convolutional transformer (MultiFormer) for hyperspectral image classification. First, the developed method utilizes multiscale spatial patches as tokens to formulate the spatial transformer and generates multiscale spatial representation of each band in each pixel. Second, the spatial representation of all the bands in a given pixel are utilized as tokens to formulate the spectral transformer and generate the multiscale spectral–spatial representation of each pixel. Besides, a modified spectral–spatial CAF module is constructed in the MultiFormer to fuse cross‐layer spectral and spatial information. Therefore, the proposed MultiFormer can capture the multiscale spectral–spatial information and provide better performance than most of other architectures for hyperspectral image classification. Experiments are conducted over commonly used real‐world datasets and the comparison results show the superiority of the proposed method.

Hypervolume (HV)-based evolutionary algorithms have been widely used to handle many-objective optimization problems. In such algorithms, HV-based environmental selection (HVES), which aims at selecting a subpopulation with the maximal HV from the current population, plays a crucial role in guiding evolution. However, the computation time of HV increases exponentially with the number of objectives, making the HVES task an expensive optimization problem. In this article, we propose an efficient surrogate-assisted greedy inclusion algorithm to deal with computationally expensive HVES tasks. It uses a lightweight surrogate model, radial basis function network, to replace the most time-consuming calculations. In addition, an$L_{1}$-norm distance-based filter is performed as a preselection operator to reduce the search space and avoid some unnecessary calculations. Considering the inevitable approximation errors of surrogate models, we also design an online sampling strategy to enhance the reliability of selected solutions. The proposed algorithm is tested on two types of datasets and compared with six state-of-the-art greedy algorithms. Experimental results show that the proposed algorithm performs excellently on most datasets.

The automatic design of soft robots characterizes as jointly optimizing structure and control. As reinforcement learning is gradually used to optimize control, the time-consuming controller training makes soft robots design an expensive optimization problem. Although surrogate-assisted evolutionary algorithms have made a remarkable achievement in dealing with expensive optimization problems, they typically suffer from challenges in constructing accurate surrogate models due to the complex mapping among structure, control, and task performance. Therefore, we propose an action inheritance-based evolutionary algorithm to accelerate the design process. Instead of training a controller, the proposed algorithm uses inherited actions to control a candidate design to complete a task and obtain its approximated performance. Inherited actions are near-optimal control policies that are partially or entirely inherited from optimized control actions of a real evaluated robot design. The action inheritance plays the role of surrogate models where its input is the structure and output is the near-optimal control actions. We also propose a random perturbation operation to estimate the error introduced by inherited control actions. The effectiveness of our proposed method is validated by evaluating it on a wide range of tasks, including locomotion and manipulation. Experimental results show that our algorithm is better than the other three state-of-the-art algorithms on most tasks when only a limited computational budget is available. Compared with the algorithm without surrogate models, our algorithm saves about half the computing cost.

Usually, data-driven multiobjective optimization problems (DD-MOPs) are indirectly solved by evolutionary algorithms through the built surrogate model which is well-trained from sample data. However, in most DD-MOPs, only a few available data can be practicably collected from real engineering experiments due to the unaffordable cost and time. The key challenge in such a DD-MOP is to prevent the serious deterioration on the accuracy of the obtained approximate Pareto front. In this article, two novel strategies, critical fitness for evolutionary algorithms and data augmentation for a surrogate model, are complementarily imposed by a generative adversarial network (GAN) to tackle with the challenges in DD-MOPs. In the critical fitness strategy, a new critical fitness, composed of the critical score from the discriminator of GAN and the prediction value of the surrogate model, is proposed to improve the accuracy of the approximate Pareto front of a DD-MOP. In the data augmentation strategy, some new samples are synthetized by the generator of GAN to build a better-trained surrogate model. As a result, the GAN concurrently serves the critical fitness strategy and the data augmentation strategy as the roles of “killing two birds with one stone.” The performance of the proposed algorithm for DD-MOPs was well-verified over 26 benchmark problems and successfully applied to discover new NdFeB materials.

Convolutional neural networks (CNNs) have shown their potential ability to extract discriminative features for hyperspectral image classification. However, traditional deep learning methods using CNNs tend to overlook the influence of complex environmental factors. These factors contribute to an increase in intraclass variance and a decrease in interclass variance, making it considerably more challenging to extract meaningful features. To overcome this problem, this work develops a novel deep intrinsic decomposition with adversarial learning, namely AdverDecom, for hyperspectral image classification to mitigate the negative impact of environmental factors on classification performance. First, we develop a generative network for hyperspectral images (HyperNet) to extract the environment-related features and category-related features from the image. Then, a discriminative network is constructed to distinguish different environmental categories. Finally, an environment-category joint learning loss is developed for adversarial learning to make the deep model learn discriminative features. Experiments are conducted over four commonly used real-world datasets and the comparison results show the superiority of the proposed method. The implementation of the proposed method could be accessed athttps://github.com/shendu-sw/Adversarial_Learning_Intrinsic_Decompositionfor the sake of reproducibility.

The dissection of hyperspectral images into intrinsic components through hyperspectral intrinsic image decomposition (HIID) enhances the interpretability of hyperspectral data, providing a foundation for more accurate classification outcomes. However, the classification performance of HIID is constrained by the model’s representational ability. To address this limitation, this study rethinks hyperspectral intrinsic image decomposition for classification tasks by introducing deep feature embedding. The proposed framework, HyperDID, incorporates the Environmental Feature Module (EFM) and Categorical Feature Module (CFM) to extract intrinsic features. Additionally, a Feature Discrimination Module (FDM) is introduced to separate environment-related and category-related features. Experimental results across three commonly used datasets validate the effectiveness of HyperDID in improving hyperspectral image classification performance. This novel approach holds promise for advancing the capabilities of hyperspectral image analysis by leveraging deep feature embedding principles. The implementation of the proposed method could be accessed soon at https://github.com/shendu-sw/HyperDID for the sake of reproducibility.

Deep neural networks (DNNs) are shown to be vulnerable to universal adversarial perturbations (UAP), a single quasi-imperceptible perturbation that deceives the DNNs on most input images. The current UAP methods can be divided into data-dependent and data-independent methods. The former exhibits weak transferability in black-box models due to overly relying on model-specific features. The latter shows inferior attack performance in white-box models as it fails to exploit the model's response information to benign images. To address the above issues, this paper proposes a novel universal adversarial attack to generate UAP with strong transferability by disrupting the model-agnostic features (e.g., edges or simple texture), which are invariant to the models. Specifically, we first devise an objective function to weaken the significant channel-wise features and strengthen the less significant channel-wise features, which are partitioned by the designed strategy. Furthermore, the proposed objective function eliminates the dependency on labeled samples, allowing us to utilize out-of-distribution (OOD) data to train UAP. To enhance the attack performance with limited training samples, we exploit the average gradient of the mini-batch input to update the UAP iteratively, which encourages the UAP to capture the local information inside the mini-batch input. In addition, we introduce the momentum term to accumulate the gradient information at each iterative step for the purpose of perceiving the global information over the training set. Finally, extensive experimental results demonstrate that the proposed methods outperform the existing UAP approaches. Additionally, we exhaustively investigate the transferability of the UAP across models, datasets, and tasks.

There is an emerging trend in the Chinese automobile industries that automakers are introducing exclusive enterprise social networks (EESNs) to expand sales and provide after-sale services. The traditional online social networks (OSNs) and enterprise social networks (ESNs), such as X (formerly known as Twitter) and Yammer, are ingeniously designed to facilitate unregulated communications among equal individuals. However, users in EESNs are naturally social stratified, consisting of both enterprise staffs and customers. In addition, the motivation to operate EESNs can be quite complicated, including providing customer services and facilitating communication among enterprise staffs. As a result, the social behaviors in EESNs can be quite different from those in OSNs and ESNs. In this work, we aim to analyze the social behaviors in EESNs. We consider the Chinese car manufacturer NIO as a typical example of EESNs and provide the following contributions. First, we formulate the social behavior analysis in EESNs as a link prediction problem in heterogeneous social networks. Second, to analyze this link prediction problem, we derive plentiful user features and build multiple meta-path graphs for EESNs. Third, we develop a novel Fast (H)eterogeneous graph (A)ttention (N)etwork algorithm for (D)irected graphs (FastHAND) to predict directed social links among users in EESNs. This algorithm introduces feature group attention at the node-level and uses an edge sampling algorithm over directed meta-path graphs to reduce the computation cost. By conducting various experiments on the NIO community data, we demonstrate the predictive power of our proposed FastHAND method. The experimental results also verify our intuitions about social affinity propagation in EESNs.

Recent work has demonstrated that pretrained transformers are overconfident in text classification tasks, which can be calibrated by the famous post-hoc calibration method temperature scaling (TS). Character or word spelling mistakes are frequently encountered in real applications and greatly threaten transformer model safety. Research on calibration under noisy settings is rare, and we focus on this direction. Based on a toy experiment, we discover that TS performs poorly when the datasets are perturbed by slight noise, such as swapping the characters, which results in distribution shift. We further utilize two metrics, predictive uncertainty and maximum mean discrepancy (MMD), to measure the distribution shift between clean and noisy datasets, based on which we propose a simple yet effective transferable TS method for calibrating models dynamically. To evaluate the performance of the proposed methods under noisy settings, we construct a benchmark consisting of four noise types and five shift intensities based on the QNLI, AG-News, and Emotion tasks. Experimental results on the noisy benchmark show that (1) the metrics are effective in measuring distribution shift and (2) transferable TS can significantly decrease the expected calibration error (ECE) compared with the competitive baseline ensemble TS by approximately 46.09%.

Physical adversarial attacks against deep neural networks (DNNs) have recently gained increasing attention. The current mainstream physical attacks use printed adversarial patches or camouflage to alter the appearance of the target object. However, these approaches generate conspicuous adversarial patterns that show poor stealthiness. Another physical deployable attack is the optical attack, featuring stealthiness while exhibiting weakly in the daytime with sunlight. In this paper, we propose a novel Reflected Light Attack (RFLA), featuring effective and stealthy in both the digital and physical world, which is implemented by placing the color transparent plastic sheet and a paper cut of a specific shape in front of the mirror to create different colored geometries on the target object. To achieve these goals, we devise a general framework based on the circle to model the reflected light on the target object. Specifically, we optimize a circle (composed of a coordinate and radius) to carry various geometrical shapes determined by the optimized angle. The fill color of the geometry shape and its corresponding transparency are also optimized. We extensively evaluate the effectiveness of RFLA on different datasets and models. Experiment results suggest that the proposed method achieves over 99% success rate on different datasets and models in the digital world. Additionally, we verify the effectiveness of the proposed method in different physical environments by using sunlight or a flashlight.

Collective swarm behavior recognition is the state-of-the-art to identify and observe a moving self-organized swarm or an object. However, because of the complex and harsh environment, current swarm behavior recognition systems remain either need large numbers of training samples or lack reliable measurements enough. In this paper, we solve the two issues in different directions, with the novel Support Vector Machines to provide guaranteed classification under the limited sample and a new conformal prediction algorithm to enhance the accuracy of the prediction. We report two classification schemes, both the simple prediction corresponding to confidence and credibility measures, and provide set prediction with any significance levels. We show why our approach performs better than other solutions through empirical studies with two benchmark datasets, both simulation and real-life swarm behavior datasets. To the best of our knowledge, we are the first to apply conformal prediction for the collective swarm behavior recognition purpose in general, and for the real-life swarm behavior in particular.

This paper introduces a martingales-protected paradigm for machine learning algorithms to solve swarm behavior classification problems. While regular machine learning algorithms, such as Neural Networks, Random Forests, and Decision Trees try to induce a general decision function for a learning task, the martingales-protected machine learning algorithms consider a set of fuzzy decision functions. The paper analyzes why the martingales-protected paradigm is well suited for most existing machine learning algorithms. These theoretical findings are supported by experiments on three swarm test collections. The experiments show substantial improvements over regular machine learning algorithms and present an empirical comparison of the new method and the classical learning approach for predictive modeling of real-life swarm behavior data. These comparisons suggest that the proposed martingale-protected method consistently performs better predictive performance than classical predictive modeling.

Full-state estimation with a limited number of sen-sors is a valuable and challenging task in monitoring and con-trolling complex physical systems. Supervised learning methods based on deep neural networks have shown excellent performance by learning the nonlinear mapping from sparse observations to global field. However, The neural network is a black box with weak explanation for physical processes, and the reconstruction performance is limited to the architecture and optimization of neural network. This paper aims to leverage the structure and laws inherent in data to reconstruct the global field by solving optimization problems instead of single network learning. We propose a unified global field reconstruction framework consisting of neural network prediction, proper orthogonal de-composition (POD), and linear optimization problem solving. The deep neural network is first trained to provide referenced global fields, which are combined with exact observations and the reference modes extracted by POD to establish a linear optimization problem. The objective of optimization problem is to superpose POD modes to satisfy the values of observations and referenced fields. The experiments conducted on fluid and thermal field reconstruction problems show that the proposed unified framework can significantly improve the reconstruction accuracy of neural networks and boost the performance of directly solving optimization problems without referenced fields.

Abstract Deep Neural Networks have been successfully applied in hyperspectral image classification. However, most of prior works adopt general deep architectures while ignore the intrinsic structure of the hyperspectral image, such as the physical noise generation. This would make these deep models unable to generate discriminative features and provide impressive classification performance. To leverage such intrinsic information, this work develops a novel deep learning framework with the noise inclined module and denoise framework for hyperspectral image classification. First, the spectral signature of hyperspectral image is modeled with the physical noise model to describe the high intra‐class variance of each class and great overlapping between different classes in the image. Then, a noise inclined module is developed to capture the physical noise within each object and a denoise framework is then followed to remove such noise from the object. Finally, the CNN with noise inclined module and the denoise framework is developed to obtain discriminative features and provides good classification performance of hyperspectral image. Experiments are conducted over two commonly used real‐world datasets and the experimental results show the effectiveness of the proposed method. The implementation of the proposed method and other compared methods could be accessed at https://github.com/shendu‐sw/noise‐physical‐framework .

Most existing adversarial attack methods against detectors involve adding adversarial perturbations to benign images to synthesiz adversarial examples. However, directly applying these methods, originally designed for natural image detectors, to optical aerial image detectors can lead to perturbations that appear unnatural and suspicious to human eyes, owing to intrinsic dissimilarities between these two types of images. Inspired by the fact that the captured optical aerial images are heavily affected by weather conditions, this paper proposes a novel method for conducting adversarial attacks against optical aerial detectors by leveraging natural weather-style perturbations. Compared to existing methods, our scheme produces more natural and stealthy adversarial examples. To enhance the practicality of the proposed method in real-world scenarios, we implement the attacks in black-box settings where only the model’s predictions are accessible. Specifically, we formulate the generation of adversarial weather perturbations in black-box as an optimization problem and effectively solve it using the Differential Evolution (DE) algorithm. Through extensive experiments, we verify the effectiveness of our method and investigate the transferability of generated adversarial examples across different models. In light of the significant generalization and effectiveness of our method, we generate and release the first dataset with adversarial weather-style perturbations based on the DOTA dataset, which we abbreviate as DOTA-W. This dataset serves as a valuable resource for evaluating and improving the robustness of optical aerial detectors. The code and dataset have been released at https://github.com/tang-agui/attADs-AWP.

Intelligent maritime transportation is one of the most promising enabling technologies for promoting trade efficiency and releasing the physical labor force. The trajectory prediction method is the foundation to guarantee collision avoidance and route optimization for ship transportation. This article proposes a bidirectional data-driven trajectory prediction method based on Automatic Identification System (AIS) spatio-temporal data to improve the accuracy of ship trajectory prediction and reduce the risk of accidents. Our study constructs an encoder-decoder network driven by a forward and reverse comprehensive historical trajectory and then fuses the characteristics of the sub-network to predict the ship trajectory. The AIS historical trajectory data of US West Coast ships are employed to investigate the feasibility of the proposed method. Compared with the current methods, the proposed approach lessens the prediction error by studying the comprehensive historical trajectory, and 60.28% has reduced the average prediction error. The ocean and port trajectory data are analyzed in maritime transportation before and after COVID-19. The prediction error in the port area is reduced by 95.17% than the data before the epidemic. Our work helps the prediction of maritime ship trajectory, provides valuable services for maritime safety, and performs detailed insights for the analysis of trade conditions in different sea areas before and after the epidemic.

Physical adversarial attacks in object detection have attracted increasing attention. However, most previous works focus on hiding the objects from the detector by generating an individual adversarial patch, which only covers the planar part of the vehicle’s surface and fails to attack the detector in physical scenarios for multi-view, long-distance and partially occluded objects. To bridge the gap between digital attacks and physical attacks, we exploit the full 3D vehicle surface to propose a robust Full-coverage Camouflage Attack (FCA) to fool detectors. Specifically, we first try rendering the nonplanar camouflage texture over the full vehicle surface. To mimic the real-world environment conditions, we then introduce a transformation function to transfer the rendered camouflaged vehicle into a photo-realistic scenario. Finally, we design an efficient loss function to optimize the camouflage texture. Experiments show that the full-coverage camouflage attack can not only outperform state-of-the-art methods under various test cases but also generalize to different environments, vehicles, and object detectors.

For the temperature field reconstruction (TFR), a complex image-to-image regression problem, the convolutional neural network (CNN) is a powerful surrogate model due to the convolutional layer's good image feature extraction ability. However, a lot of labeled data is needed to train CNN, and the common CNN can not quantify the aleatoric uncertainty caused by data noise. In actual engineering, the noiseless and labeled training data is hardly obtained for the TFR. To solve these two problems, this paper proposes a deep Monte Carlo quantile regression (Deep MC-QR) method for reconstructing the temperature field and quantifying aleatoric uncertainty caused by data noise. On the one hand, the Deep MC-QR method uses physical knowledge to guide the training of CNN. Thereby, the Deep MC-QR method can reconstruct an accurate TFR surrogate model without any labeled training data. On the other hand, the Deep MC-QR method constructs a quantile level image for each input in each training epoch. Then, the trained CNN model can quantify aleatoric uncertainty by quantile level image sampling during the prediction stage. Finally, the effectiveness of the proposed Deep MC-QR method is validated by many experiments, and the influence of data noise on TFR is analyzed.

Recent studies show that deep neural networks are vulnerable to adversarial attacks in the form of subtle perturbations to the input image, which leads the model to output wrong prediction. Such an attack can easily succeed by the existing white-box attack methods, where the perturbation is calculated based on the gradient of the target network. Unfortunately, the gradient is often unavailable in the real-world scenarios, which makes the black-box adversarial attack problems practical and challenging. In fact, they can be formulated as high-dimensional black-box optimization problems at the pixel level. Although evolutionary algorithms are well known for solving black-box optimization problems, they cannot efficiently deal with the high-dimensional decision space. Therefore, we propose an approximated gradient sign method using differential evolution (DE) for solving black-box adversarial attack problems. Unlike most existing methods, it is novel that the proposed method searches the gradient sign rather than the perturbation by a DE algorithm. Also, we transform the pixel-based decision space into a dimension-reduced decision space by combining the pixel differences from the input image to neighbor images, and two different techniques for selecting neighbor images are introduced to build the transferred decision space. In addition, six variants of the proposed method are designed according to the different neighborhood selection and optimization search strategies. Finally, the performance of the proposed method is compared with a number of the state-of-the-art adversarial attack algorithms on CIFAR-10 and ImageNet datasets. The experimental results suggest that the proposed method shows superior performance for solving black-box adversarial attack problems, especially nontargeted attack problems.

Various evolutionary algorithms (EAs) have been proposed to address feature selection (FS) problems, in which a large number of fitness evaluations are needed. With the rapid growth of data scales, the fitness evaluation becomes time consuming, which makes FS problems expensive optimization problems. Surrogate-assisted EAs (SAEAs) have been widely used to solve expensive optimization problems. However, the SAEAs still face difficulties in solving expensive FS problems due to their high-dimensional discrete decision variables. To address this issue, we propose an SAEA with parallel random grouping for expensive FS problems, in which three main components consist. First, a constraint-based sampling strategy is proposed, which considers the influence of the constraint boundary and the number of selected features. Second, a high-dimensional FS problem is randomly divided into several low-dimensional subproblems. Surrogate models are then constructed in these low-dimensional decision spaces. After that, all the subproblems are optimized in parallel. The process of random grouping and parallel optimization continues until the termination condition is met. Finally, a final solution is chosen from the best solution in the historical search and the best solution in the last population using a random, distance-, or voting-based method. Experimental results show that the proposed algorithm generally outperforms traditional, ensemble, and evolutionary FS methods on 14 datasets with up to 10 000 features, especially when the required number of real fitness evaluations is limited.

This article proposes a new network approach toward the implementation of Takagi–Sugeno (T–S) fuzzy models referred to as disjunctive fuzzy neural networks (DJFNNs). The proposed DJFNN involves a novel network architecture and a greedy learning algorithm. Being different from the existing grid-based and clustering-based network architectures, the proposed architecture adds an OR neural layer positioned between the fuzzification layer and the rule layer. In this way, the implied constraint between the number of rules and the number of fuzzy labels is excluded so that a curse of dimensionality can be overcome and more interpretable models are formed. Furthermore, inspired by the core algorithm for building a decision tree, a top–down, nonbacktracking, and greedy algorithm is proposed to learn the unknown parameters of the networks. The input space splits into smaller and smaller subspace along the predefined fuzzy grids in a supervised manner meanwhile the associated conditions of the T–S fuzzy model are identified. The greedy algorithm is applicable to high-dimensional problems since there is no exponential growth in time or space as the dimensionality increases. The new network architecture and greedy learning algorithm make the proposed DJFNN a regression model of high interpretability and good prediction capability, particularly suitable for solving the high-dimensional problems. The DJFNN was experimented with using a synthetic dataset and 28 real-world datasets and compared with classical and state-of-the-art methods through nonparametric statistical tests. The results confirmed the effectiveness of the DJFNN in terms of accuracy, interpretability, and computational cost.

For complex system design [e.g., satellite layout optimization design (SLOD)] in practical engineering, when launching a new optimization instance with another parameter configuration from the intuition of designers, it is always executed from scratch which wastes much time to repeat the similar search process. Inspired by transfer learning which can reuse past experiences to solve relevant tasks, many researchers pay more attention to explore how to learn from past optimization instances to accelerate the target one. In real-world applications, there have been numerous similar source instances stored in the database. The primary question is how to measure the transferability from numerous sources to avoid the notorious negative transferring. To obtain the relatedness between source and target instance, we develop an optimization instance representation method named centroid distribution, which is by the aid of the probabilistic model learned by elite candidate solutions in estimation of distribution algorithm (EDA) during the evolutionary process. Wasserstein distance is employed to evaluate the similarity between the centroid distributions of different optimization instances, based on which, we present a novel framework called multisource selective transfer optimization with three strategies to select sources reasonably. To choose the suitable strategy, four selection suggestions are summarized according to the similarity between the source and target centroid distribution. The framework is beneficial to choose the most suitable sources, which could improve the search efficiency in solving multiobjective optimization problems. To evaluate the effectiveness of the proposed framework and selection suggestions, we conduct two experiments: 1) comprehensive empirical studies on complex multiobjective optimization problem benchmarks and 2) a real-world SLOD problem. Suggestions for strategy selection coincide with the experiment results, based on which, we propose a mixed strategy to deal with the negative transfer in the experiments successfully. The results demonstrate that our proposed framework achieves competitive performance on most of the benchmark problems in convergence speed and hypervolume values and performs best on the real-world applications among all the comparison algorithms.

The satellite layout optimization design (SLOD) problem is a kind of three-dimensional layout problems with complex performance constraints and known as a NP-hard problem. To solve SLOD problems efficiently and effectively, two types of hybrid optimization algorithm based on differential evolution (DE) are proposed in this paper. Concerning the design requirements of satellite attitude control subsystem, the SLOD problem is formulated, aiming to improve the overall mass characteristics of satellite. To explore the layout design space globally, the DE algorithm is utilized as the main framework of the proposed hybrid algorithm. Then in order to improve the local exploitation capability and algorithm robustness, sequential quadratic programming (SQP), as a gradient-based method, is combined with DE in two unique ways, comprising two types of hybrid algorithm. In the first type of hybrid algorithm (denoted by DESQP), SQP is performed when iteration process of DE has finished and only the final solution of DE is used as the initial point of SQP, the purpose of which is to locate the most promising area of optimum with DE first and then make a rapid exploitation around the quasi-optimum. In the second type of hybrid algorithm (denoted by DESQPDE), SQP is performed in the specific iteration of DE and all the current-generation population individuals are used as the initial points, the purpose of which is to accelerate the evolution process while holding the diversity of the population and to enhance the robustness. Finally, the efficacy and robustness of the proposed hybrid algorithms are compared with classical DE and also validated by two three-dimensional satellite layout cases with 14 and 40 components, respectively.

To properly define satellite lifetime so as to maximize economic net benefit, the lifetime optimization method is studied in this paper based on discrete cross entropy (CE) method. Firstly, the influences of lifetime on satellite system design, cost and revenue are studied and the disciplinary models are developed. Then the lifetime optimization problem is formulated, which is a typical discrete optimization problem with the discrete lifetime measured in years and the combination of the subsystem components as design variables. To efficiently solve this problem, the CE method is used as the optimization solver, and a convolution based smoothing strategy is proposed to enhance the space exploration capability and algorithm robustness. The efficacy of the proposed method is demonstrated in a 19 dimensional satellite lifetime design problem, which also verifies the importance of lifetime optimization in enhancing the satellite economic benefit.

No doubt that the proper design of satellite system reliability is one of the most important issues in satellite system engineering. In order to optimize the satellite reliability, the reliability optimization method is studied in this paper based on compression algorithm, Bayesian Network (BN) and Particle Swarm Optimization (PSO). Firstly, the improved compression inference algorithm (ICIA) is proposed to model the BN reliability model of multistate system based on the existing compression and inference algorithms. Then, the reliability optimization problem is formulated, which is a typical discrete optimization problem with the discrete reliability measured in the number of each component's units as design variables. The process of optimization, based on the BN reliability model of multistate satellite system will be achieved by PSO. The efficacy of the proposed method is demonstrated in a microsatellite system reliability optimization, which verifies the importance of satellite reliability optimization in decreasing the cost of satellite development.

Accessibility evaluation is the primary step for the selection of the visiting target before the implementation of the main-belt asteroids manned exploration mission. Optimal transfer velocity increments from the earth to the asteroids and back to the earth must be obtained for the accessibility evaluation and analysis. Optimizing them one by one for all the candidate asteroids is extremely inefficient because of the great time consuming caused by the optimization process. In this paper, a learning-based method is applied to overcome this issue. Some optimal solutions are first produced as the training samples and the estimation model is trained to quickly obtain the optimal velocity increments of all the candidate asteroids. The experimental result shows the superiority of the learning-based method for solving this problem. No more than 1/50 simulation time is enough compared with the optimization-based method with an acceptable average relative error of 1.2%.

Multidisciplinary design optimization (MDO) under uncertainty is increasingly being recognized in improving the performance, safety, and reliability of aerospace vehicles. However, the solution process is still challenging, especially in multi-objective optimizations. In this study, a multi-objective alliance algorithm (MOAA) is employed and corresponding computational heuristics are presented, including system decoupling strategy, active subspaces, and surrogate model. Both reliability-based design optimization (RBDO) and robust design optimization (RDO) are considered to prove the efficacy of the proposed approach, which is exemplified by the conceptual design of a small satellite mission for the Moon imaging. Among multiple system uncertainties and closely coupled disciplines, the approach exhibits high effectiveness and strong adaptability at considerably less cost, thus providing a potential approach to solving widely exiting MDO problems of aerospace vehicles.

Surrogate modeling is a common method for computationally intensive engineering design optimization problems. For lack of prior knowledge, it is difficult to decide which surrogate is more suitable for approximation. In order to take full advantage of multiple surrogates, ensembles of surrogate models have been gradually focused on. However, the current ensemble methods do not consider the relation between weights and error variation of component surrogates, which is helpful for improving the prediction accuracy. In this paper, a novel point-wise weighted ensemble approach of surrogate models is proposed, which combines error classification and nearest neighbour choosing into point-wise weights computing. First, the leave-one-out cross validation errors of each component surrogate are classified into several error levels based on unsupervised learning. Second, nearest neighbour training points of each test point are selected according to distance sorting. Considering local error of each test point, only nearest neighbour training points which achieve a user-defined error level of each component surrogate are selected. Finally, based on the above steps, the point-wise weights of each test point are computed for the component surrogates. Experiments show that the proposed ensemble method outperforms the state-of-the-art methods in three benchmark function examples.

Radial basis function neural networks (RBFNNs) are widely used in nonlinear function approximation. One of the challenges in RBFNN modeling is determining how to effectively optimize width parameters to improve approximation accuracy. To solve this problem, a width optimization method, concurrent subspace width optimization (CSWO), is proposed based on a decomposition and coordination strategy. This method decomposes the large-scale width optimization problem into several subspace optimization (SSO) problems, each of which has a single optimization variable and smaller training and validation data sets so as to greatly simplify optimization complexity. These SSOs can be solved concurrently, thus computational time can be effectively reduced. With top-level system coordination, the optimization of SSOs can converge to a consistent optimum, which is equivalent to the optimum of the original width optimization problem. The proposed method is tested with four mathematical examples and one practical engineering approximation problem. The results demonstrate the efficiency and robustness of CSWO in optimizing width parameters over the traditional width optimization methods.

The design of radial basis function widths of Radial Basis Function Neural Network (RBFNN) is thoroughly studied in this paper. Firstly, the influence of the widths on performance of RBFNN is illustrated with three simple function approximation experiments. Based on the conclusions drawn from the experiments, we find that two key factors including the spatial distribution of the training data set and the nonlinearity of the function should be considered in the width design. We propose to use Euclidean distances between center nodes and the second derivative of function to measure these two factors respectively. Secondly, a two step method is proposed to design the widths based on the information about the aforementioned two key factors obtained from comprehensive analysis of the given training data set. In the first step the data set spatial distribution features are analyzed according to the Euclidean distances between the data points, and the second derivative of each center node is estimated with finite difference approximation method. Based on the analysis an initial design of the widths is given with a heuristic equation. In the second step optimization techniques are used to optimize the widths which can effectively find the optimum with the good initial baseline. Thirdly, one mathematical example is taken to verify the efficiency of the proposed method, and followed by conclusions.