Generative Adversarial Networks (GANs) have demonstrated remarkable success in high-quality image synthesis, with StyleGAN and its successor, StyleGAN2, achieving state-of-the-art performance in terms of realism and control over generated features. However, the large number of parameters and high floating-point operations per second (FLOPs) hinder real-time applications and scalability, posing challenges for deploying these models in resource-constrained environments such as edge devices and mobile platforms. To address this issue, we propose Evolutionary Channel Pruning for StyleGANs (ECP-StyleGANs), a novel algorithm that leverages evolutionary algorithms to compress StyleGAN and StyleGAN2 while maintaining competitive image quality. Our approach encodes pruning configurations as binary masks on the model's convolutional channels and iteratively refines them through selection, crossover, and mutation. By integrating carefully designed fitness functions that balance model complexity and generation quality, ECP-StyleGANs identifies optimally pruned architectures that reduce computational demands without compromising visual fidelity, achieving approximately a 4 × reduction in FLOPs and parameters, while maintaining visual fidelity with only a slight increase in FID (Fréchet Inception Distance) compared to the original un-pruned model. This study should be interpreted as a preliminary step towards the formulation and management of the generative AI pruning problem as a multi-objective optimisation task, aimed at enhancing the trade-off between model efficiency and image quality, thereby making large deep models more accessible for real-world applications such as edge devices and resource-constrained environments.

Many real-world problems involve optimizing numerous decision variables and are expensive to evaluate, known as large-scale expensive optimization problems (LSEOPs). While surrogate-assisted evolutionary algorithms have proven effective for expensive problems, training proper models for LSEOPs remains challenging due to insufficient training data. In this paper, we adopt the divide-and-conquer approach, decomposing LSEOPs into lower-dimensional sub-problems and constructing models for sub-problems, and introduce a multi-view synthetic sampling technique for new sample selection. Specifically, we propose sorting all evaluated solutions in an ascending order and dividing them into intervals, from which data are sampled to obtain informative training data for models. The population for the LSEOP is updated by employing cooperative environmental selections on the population, formed by recombining all renewed populations for sub-problems to balance exploration and exploitation. Finally, a solution is selected among the current population for the true evaluation based on its multi-view performance predicted across all sub-problems. Results on CEC’2013 benchmark problems show the effectiveness and efficiency of our proposed method compared to three prevalent large-scale expensive optimization algorithms. Additionally, results on 2000-dimensional CEC’2010 benchmark problems and a 1200-dimensional real-world problem demonstrate encouraging scalability and robustness of the proposed method for addressing higher-dimensional problems.

Federated Bayesian Optimization (FBO) enables collaborative optimization across distributed data sources without direct data exchange, addressing privacy concerns in domains such as healthcare and manufacturing. However, existing FBO approaches often suffer from high communication overhead and computational costs due to the complexity of sharing and updating Gaussian Process (GP) models across federated clients. This paper presents a novel framework that combines symbolic regression (SR) with GPs to create lightweight surrogate models for federated black-box optimization. Our approach employs SR to generate compact mathematical expressions for client-server communication while utilizing local GPs to model uncertainty, significantly reducing bandwidth requirements and computational complexity. The framework incorporates a Lower Confidence Bound sampling strategy that combines SR predictions with GP posterior distributions to balance exploration and exploitation. Experimental results demonstrate the reliability and efficacy of our proposed method on benchmark problems.

Neural Architecture Search (NAS) has emerged as a promising approach to automating the discovery of optimal neural network architectures. However, the computational cost of evaluating candidate architectures through full training presents a significant barrier to efficient search. While surrogate models can accelerate this process by predicting network performance, they struggle to accurately model the vast architecture search space when working with limited training data. To mitigate this challenge, we propose a Multi-Task, Multi-Surrogate Assisted Evolutionary NAS framework (MT-MSAENAS) that combines multiple surrogate models to enhance search efficiency. To fully exploit the limited training data, MT-MSAENAS constructs both strong and weak surrogate predictors, a global model (strong) that captures overall search space patterns and a local model (weak) that specializes in promising regions. Based on the two surrogate model, MT-MSAENAS employs evolutionary multi-tasking optimization by treating the strong and weak model-assisted search as two related optimization tasks to facilitate knowledge transfer between these models and improve the search efficiency. Experiments on the NAS-Bench101 and NAS-Bench201 search spaces show that the proposed algorithm outperforms state-of-the-art methods in architecture search.

Image Aesthetics Assessment (IAA) is a challenging task due to its subjective nature and expensive manual annotations. Recent large-scale vision-language models, such as Contrastive Language-Image Pre-training (CLIP), have shown their promising representation capability for various downstream tasks. However, the application of CLIP to resource-constrained and low-data IAA tasks remains limited. While few attempts to leverage CLIP in IAA have mainly focused on carefully designed prompts, we extend beyond this by allowing models from different domains and with different model sizes to acquire knowledge from CLIP. To achieve this, we propose a unified and flexible two-phase CLIP-based Semi-supervised Knowledge Distillation (CSKD) paradigm, aiming to learn a lightweight IAA model while leveraging CLIP’s strong generalization capability. Specifically, CSKD employs a feature alignment strategy to facilitate the distillation of heterogeneous CLIP teacher and IAA student models, effectively transferring valuable features from pre-trained visual representations to two lightweight IAA models, respectively. To efficiently adapt to downstream IAA tasks in a low-data regime, the two strong visual aesthetics learners then conduct distillation with unlabeled examples for refining and transferring the task-specific knowledge collaboratively. Extensive experiments demonstrate that the proposed CSKD achieves state-of-the-art performance on multiple widely used IAA benchmarks. Furthermore, analysis of attention distance and entropy before and after feature alignment shows the effective transfer of CLIP’s feature representation to IAA models, which not only provides valuable guidance for the model initialization of IAA but also enhances the aesthetic feature representation of IAA models. Code will be made publicly available.

This reproducibility companion paper provides implementation details of our paper ''Learning differentiable particle filter on the fly''[10] presented at the 57th Asilomar Conference on Signals, Systems, and Computers. We provide detailed documentation to replicate our research, which proposes a differentiable particle filter capable of online learning. This paper includes our Python code repository, experimental configurations, dataset description, and step-by-step instructions to reproduce the results. By sharing these resources, we aim to encourage open source and further research in this direction.

Black-box optimization problems are commonly seen in the real world, ranging from experimental design to hyperparameter tuning of machine learning models. In numerous scenarios, addressing a collection of similar data-driven black-box optimization tasks distributed on multiple clients not only raises privacy concerns but also suffers from non-independent and identically distributed (non-IID) data, seriously deteriorating the optimization performance. To address the above challenges, this article focuses on handling non-IID data in federated data-driven many-task optimization. To construct a high-quality global surrogate by robustly aggregating the local models, the server first fits a Gaussian distribution for each model parameter upon receiving local parameters, from which an ensemble model can be sampled. To reduce the communication cost and provide a generalized global model, a student surrogate model is derived by means of knowledge distillation from the ensemble. In addition, each client is allowed to retain both local and global models, so that the mean and variance of the predictions can be used to guide the selection of new samples. Experimental results demonstrate the reliability and efficacy of our proposed method on both benchmark problems and a real machine learning problem in the presence of non-IID data.

In surrogate-assisted evolutionary optimization, privacy-preservation and trusted data sharing has become an increasingly important concern, especially in scenarios involving distributed sensitive data. Existing privacy-preserving surrogate-assisted evolutionary optimization algorithms heavily rely on the basic federated learning framework. However, recent findings have revealed possible vulnerabilities within this framework, including susceptibility to adversarial threats like gradient leakage and inference attacks. To address the above challenges and enhance privacy protection, this paper proposes to protect the raw data by applying a differentially private stochastic gradient descent method to train surrogate models. A differential evolution operator is designed to generate personalized new samples for multiple clients based on promising and additional auxiliary samples, avoiding the exposure of online newly generated data. Moreover, a similarity-based aggregation algorithm is integrated to effectively construct the global surrogate model. A rigorous security analysis is provided to further validate the effectiveness of the proposed method in privacy protection. Experimental results show that the proposed method exhibits remarkable optimization performance on a set of synthetic problems with federated settings while maintaining the data privacy.

Traditional large-scale evolutionary algorithms are limited in their ability to solve certain real-world applications with high-dimensional, closed-box, and computationally expensive objectives due to their need for numerous objective evaluations. Surrogate-assisted evolutionary algorithms (SAEAs) have shown effective for expensive closed-box optimization by relying on inexpensive surrogate models. However, large-scale optimization remains challenging for SAEAs due to the exponentially growing search space and the presence of multiple local optima, resulting in difficulty in training a proper model due to the lack of samples. To address these challenges, we propose constructing an initial surrogate model on randomly selected dimensions and calculating a Gaussian distribution for each sampled dimension. The surrogate then provides predictions when perturbing each sampled dimension by sampling from the distribution, enabling the identification of the most important variables for constructing an active subproblem to reduce the search space. A secondary surrogate model, built for the active subproblem, guides the offspring generation and environmental selection for a modified particle swarm optimization algorithm to effectively explores the subspace while escaping local optima in large-scale problems. Experimental results on CEC’2013 and CEC’2010 benchmark problems show that the proposed method outperforms state-of-the-art algorithms in addressing large-scale expensive optimization problems. The efficiency of the proposed method is further verified on CEC’2010 benchmark problems extended to 2000 dimensions.

Surrogate-assisted evolutionary algorithms (SAEAs) have emerged as a promising approach to addressing expensive and black-box problems. Most existing SAEAs leverage regression models to predict the objective values, reducing the use of true objective functions. However, these methods focus on learning the mapping from the decision space to the objective space and may fail to reveal the relationship between solutions in the decision space. Recently, graph neural networks (GNNs) have attracted increased attention due to their powerful ability to expose sample interaction. In this paper, we propose employing a graph neural network for learning embeddings of solutions in the decision space, followed by a classification task aimed at predicting dominance relationships between solutions in the objective space and a regression task for obtaining the estimated fitness values. To this end, we generate a graph at each generation to represent the topology relationship between solutions in the decision space, where nodes represent solutions, and edges are added depending on the Euclidean distances between nodes. In addition, a new acquisition function that adaptively weights the predictions on objective values and dominance relationships is proposed to effectively identify new samples. The performance of the proposed method is examined by extensive empirical studies on a widely used test suite in comparison to its peer algorithms, and the results confirm the effectiveness of the proposed method.

With the development of deep learning, many PolSAR image classification methods based on deep learning have shown impressive performance. Most of these methods rely on manually designed neural networks, which requires a lot of professional knowledge. In response to this issue, this study proposes a complex-valued convolutional neural network architecture search based on evolutionary algorithm for PolSAR image classification. The complex-valued convolutional neural network can extract the complex-valued characteristics of PolSAR images directly. The architecture search for complex-valued convolutional neural network is transformed into an evolutionary population-based optimization, where classification accuracy is the fitness function. Furthermore, a two-steps selection strategy is designed to reduce the architected complexity while ensuring high classification accuracy. The experimental results on two different PolSAR datasets show that the automatic-designed complex-valued convolutional neural network have superior classification performance.

Bayesian optimization is a powerful surrogate-assisted algorithm for solving expensive black-box optimization problems. While Bayesian optimization was developed for centralized optimization, the availability of massive distributed data has attracted increased interests in exploring federated Bayesian optimization that can use data on multiple clients without leaking the raw data. However, existing federated Bayesian optimization (FBO) approaches assume that either all clients jointly solve the same optimization task, or only one client solves one target optimization task by transferring knowledge from others in a federated way, making them unsuited for many real-world applications. In this paper, we consider FBO for the scenario where multiple related local black-box tasks associated with different clients are jointly optimized by sharing knowledge between tasks without leaking the data privacy. An efficient federated many-task Bayesian optimization framework is proposed to address not independent and identically distributed (non-IID) data while protecting the data privacy in the federated setting. A novel federated knowledge transfer paradigm is developed for dynamic many-task model aggregation according to a dissimilarity matrix. The dissimilarity is measured based on the rank of the predictions and only the hyperparameters in the local Gaussian process models are shared. In addition, a federated ensemble acquisition function is constructed by integrating the predictions of two surrogates using the global and local hyperparameters, respectively, to effectively search for the optimal solution. Experimental results show that our proposed method has reliable performance on both benchmark problems and a real machine learning problem also in the presence of non-IID data.

No abstract available.

Multiobjective optimization problems whose objectives have different evaluation costs are commonly seen in the real world. Such problems are now known as multiobjective optimization problems with heterogeneous objectives (HE-MOPs). So far, however, only a few studies have been reported on addressing HE-MOPs, and most of them focus on biobjective problems with one fast objective and one slow objective. In this work, we aim to deal with HE-MOPs having more than two black-box and heterogeneous objectives. To this end, we develop a multiobjective Bayesian evolutionary optimization (BEO) approach to HE-MOPs that can alleviate search biases resulting from the different numbers of function evaluations allowed for the cheap and expensive objectives, which is achieved by designing a new acquisition function that penalizes the search bias toward the fast objectives, thereby achieving a balance between convergence and diversity. In addition, to make the best use of the different amounts of training data while avoiding increasing the computational cost, an ensemble consisting of two Gaussian processes is constructed for each cheap objective, one trained on the data collected before the Bayesian optimization starts, and the other on those evaluated during the BEO. Empirical studies on widely used multi-/many-objective benchmark problems whose objectives are assumed to be heterogeneously expensive demonstrate that the proposed algorithm is able to find high-quality solutions for HE-MOPs compared with the state-of-the-art methods.

Deep neural networks have achieved remarkable success in various fields, leading to an increasing demand for automatic design of high-quality network architectures instead of manual design. Neural architecture search offers a solution to automatic tuning of network architectures and associated hyperparameters. Early work on NAS assumes that all the training data is stored centrally on a single device, which is unrealistic in many real-world applications. In recent years, there has been a surge in research on federated neural architecture search due to the growing amount of data on edge devices and the increasing concerns over security and privacy protection. It allows several parties to collaboratively search for an optimal model without leaking the privacy of the data. However, existing work on federated NAS transmits the entire weight parameters of the candidate architectures between the clients and the server, leading to high communication costs and potential risks of leaking raw data. To fill this gap, this paper proposes a federated Bayesian optimization NAS approach called FL-BONAS. Instead of uploading the original weight values to the server during each round, the clients cooperatively train an ensemble surrogate model as the neural predictor, and each client searches for its own architectures via Bayesian optimization without revealing either the local raw data or the architectures. To the best of our knowledge, this is the first work to train an ensemble model in Bayesian optimization for NAS under federated learning scenarios. FL-BONAS significantly improves the data efficiency while protecting privacy. The experimental results show that the proposed FL-BONAS can achieve better performance and protect raw data compared with centralized NAS.

Most existing multiobjective evolutionary algorithms (MOEAs) implicitly assume that each objective function can be evaluated within the same period of time. Typically. this is untenable in many real-world optimization scenarios where evaluation of different objectives involves different computer simulations or physical experiments with distinct time complexity. To address this issue, a transfer learning scheme based on surrogate-assisted evolutionary algorithms (SAEAs) is proposed, in which a co-surrogate is adopted to model the functional relationship between the fast and slow objective functions and a transferable instance selection method is introduced to acquire useful knowledge from the search process of the fast objective. Our experimental results on DTLZ and UF test suites demonstrate that the proposed algorithm is competitive for solving bi-objective optimization where objectives have non-uniform evaluation times.

Gaussian processes (GPs) are widely used in surrogate-assisted evolutionary optimization of expensive problems mainly due to the ability to provide a confidence level of their outputs, making it possible to adopt principled surrogate management methods, such as the acquisition function used in the Bayesian optimization. Unfortunately, GPs become less practical for high-dimensional multiobjective and many-objective optimization as their computational complexity is cubic in the number of training samples. In this article, we propose a computationally efficient dropout neural network (EDN) to replace the Gaussian process and a new model management strategy to achieve a good balance between convergence and diversity for assisting evolutionary algorithms to solve high-dimensional multiobjective and many-objective expensive optimization problems. While the conventional dropout neural network needs to save a large number of network models during the training for calculating the confidence level, only one single network model is needed in the EDN to estimate the fitness and its confidence level by randomly ignoring neurons in both training and testing the neural network. Extensive experimental studies on benchmark problems with up to 100 decision variables and 20 objectives demonstrate that, compared to state of the art, the proposed algorithm is not only highly competitive in performance but also computationally more scalable to high-dimensional many-objective optimization problems. Finally, the proposed algorithm is validated on an operational optimization problem of crude oil distillation units, further confirming its capability of handling expensive problems given a limited computational budget.

Despite the success of evolutionary algorithms (EAs) for solving multi-objective problems, most of them are based on the assumption that all objectives can be evaluated within the same period of time. However, in many real-world applications, such an assumption is unrealistic since different objectives must be evaluated using different computer simulations or physical experiments with various time complexities. To address this issue, a surrogate assisted evolutionary algorithm along with a parameter-based transfer learning (T-SAEA) is proposed in this work. While the surrogate for the cheap objective can be updated on sufficient training data, the surrogate for the expensive one is updated by either the training data set or a transfer learning approach. To find out the transferable knowledge, a filter-based feature selection algorithm is used to capture the pivotal features of each objective, and then use the common important features as a carrier for knowledge transfer between the cheap and expensive objectives. Then, the corresponding parameters in the surrogate models are adaptively shared to enhance the quality of the surrogate models. The experimental results demonstrate that the proposed algorithm outperforms the compared algorithms on the bi-objective optimization problems whose objectives have a large difference in computational complexities.