In multimodal multi-objective optimization problems (MMOPs), multiple solutions on different Pareto optimal solution sets (PSs) are mapped to the same point on the Pareto front. Considering these different solutions can provide users with richer decisions, the search of multiple PSs is crucial when solving MMOPs. To this end, many multimodal multi-objective evolutionary algorithms (MMOEAs) often employ intricate mechanisms to maintain the diversity of the offspring in mating selection, but ignore to learn PSs. In this paper, a data-driven evolutionary algorithm based on inductive graph neural networks (DEA-IGNN) is proposed to solve MMOPs, which successfully learns the PSs topology by the graph structure to generate offspring with good performance. Specifically, a graph topology construction method based on Euclidean distance in the decision space is designed. It determines the neighborhood by calculating the Euclidean distance of individuals in the decision space and establishes the topological relationships to construct the graphs representing of population distribution. On this basis, a model based on inductive graph neural networks is constructed to assist offspring reproduction, which can learn unknown nodes by sampling and aggregating existing information. Moreover, a data-driven reproduction strategy is proposed to predict offspring with the good diversity and convergence, which uses the traditional variation operators to generate training data and adopts these data to train the model. The proposed DEA-IGNN is implemented and compared with eleven competitive MMOEAs on three test suites and a practical problem. The experimental results show that DEA-IGNN has good performance.

Many-task optimization typically assumes that all data is available on a single device without taking into account privacy concerns. Federated many-task optimization, which involves using data from multiple devices, faces challenges due to the need to balance between knowledge sharing and privacy protection. To tackle the above challenge, we introduce a federated Bayesian optimization algorithm that optimizes the global acquisition function, which is approximated through a global classifier constructed by aggregating the local classifiers from all clients. In the proposed algorithm, a local neural network classifier is constructed on each client to learn the pairwise rank relationship between two solutions, which is trained on samples generated by the local acquisition function. Then, the parameters of local classifiers are transmitted to the server to construct a global classifier, based on which a competitive particle swarm optimizer is employed to optimize the global acquisition function without explicitly building it. In this way, the proposed algorithm can perform optimization using data distributed on multiple clients without sharing them. To more effectively deal with many-task optimization problems, the similarity between the tasks is measured according to the Euclidean distance between the weight vectors of the local classifiers, such that clients having similar tasks will share a common global classifier. To validate the performance of the proposed algorithm, we conduct empirical studies on a set of single-task and many-task benchmark problems. The experimental results demonstrate that our algorithm is highly competitive, showcasing its efficiency and effectiveness compared with the state-of-the-art privacy-preserving Bayesian optimization algorithms.

Surrogate-assisted evolutionary algorithms (SAEAs) have garnered significant attention for addressing expensive multiobjective optimization problems. Most existing SAEAs, however, still rely on conventional genetic operators in reproduction, which is inefficient in generating promising candidate solutions. To address the above issue, this article presents a learning-based generative model that replaces crossover and mutation and learns to conduct multiobjective search for expensive multiobjective optimization problems. The key idea is to design an attention-enhanced convolutional residual network with the assistance of surrogate model for offspring generations. The proposed framework employs a generative model to produce promising solutions for each decomposed subproblem based on the Tchebycheff metric, while a surrogate model assists in optimizing the generative model's hyperparameters through an online learning process. We demonstrate the efficacy of our learning-based multiobjective generative model (LMOGM) on DTLZ, ZDT, and WFG benchmark function suites, varying in dimensions from 30 to 200, as well as through a practical application involving the geothermal energy extraction design optimization. Experimental results highlight the superior performance of the proposed approach when compared to traditional evolutionary algorithms and state-of-the-art surrogate-assisted multiobjective evolutionary algorithms (MOEAs).

Constrained multiobjective optimization problems (CMOPs) are widely present in the real world, and the key difficulty in solving CMOPs lies in effectively handling the interplay between satisfying constraints and optimizing conflicting objectives. Many constrained multiobjective optimization evolutionary algorithms (CMOEAs) have been proposed for solving CMOPs, among which the evolutionary multitasking (EMT)-based CMOEAs have demonstrated notable performance. However, most existing EMT-based CMOEAs focus on designing reasonable auxiliary tasks but do not address how to effectively facilitate knowledge transfer between multiple tasks. Building on these considerations, this article proposes a multiknowledge adaptive transfer framework that incorporates two novel categories of knowledge: synchronization knowledge in the decision space and evolutionary direction knowledge. This approach can learn more effectively from other populations compared to traditional EMT-based CMOEAs. Moreover, to prevent the auxiliary population from failing to provide effective value to the main population and wasting computational resources, a reward-based auxiliary value assessment strategy is proposed. Whether to terminate the reproduction of the auxiliary population is evaluated by the value of the auxiliary population to the main population. Based on these two improvements, this article introduces a novel multiknowledge adaptive transfer framework for constrained multiobjective optimization (MKATCMO). The experimental results on 33 benchmark test problems and 28 practical engineering application problems demonstrate that the proposed MKATCMO has excellent performance compared with nine state-of-the-art (SOTA) CMOEAs.

As the dimensionality of the decision space in multi-objective optimization problems increases, the decision space expands exponentially, presenting significant challenges to the search efficiency of traditional multi-objective evolutionary algorithms in large-scale multi-objective optimization problems. To quickly locate promising search regions in the vast decision space, this paper proposes utilizing denoising diffusion probabilistic models to generate promising solutions, based on which a novel elite-guided large-scale multi-objective evolutionary algorithm is introduced. Specifically, in our proposed method, the population is divided into elite and poor solutions, with each poor solution paired with an elite solution. The elite solutions serve as generation targets, and their paired poor solutions act as conditions during the training of the generative model. Our approach allows the model to not only capture the distribution of elite solutions but also effectively model the evolutionary trajectory from poor solutions to elite solutions. The entire population is used as conditions, and the trained generative model generates ideal positions, which are then updated to produce offspring solutions. Experimental results on large-scale multi-objective benchmark functions demonstrate that the proposed algorithm outperforms four state-of-the-art large-scale multi-objective evolutionary algorithms.

This paper addresses the challenge of identifying similarities between different optimization tasks, which is crucial for enhancing transfer learning and automated optimization systems. Traditional rule-based methods often fail to capture the complexity of problems, while existing data-driven approaches lack comprehensive feature representation and generalization across domains. Moreover, there is a severe lack of training data when applying deep learning strategies. To overcome these limitations, we propose a novel model based on contrastive learning for optimization task similarity recognition. Our approach integrates information from the decision space, objective space, and derivative space, creating a unified representation framework inspired by image data formats. We employ a convolutional neural network to extract task features and utilize contrastive learning to measure task similarity. Experimental results demonstrate the model’s effectiveness in generalizing to new optimization tasks and its sensitivity to task differences. We conducted experiments on 40- and 60-dimensional problems, where sampling only 5 times the dimensionality of data points was sufficient for distinction. The proposed method not only provides a comprehensive representation of optimization tasks but also enhances the model’s generalization performance.

Multi-objective optimization (MOO) exists extensively in machine learning, and aims to find a set of Pareto-optimal solutions, called the Pareto front, e.g., it is fundamental for multiple avenues of research in federated learning (FL). Pareto-Front Learning (PFL) is a powerful method implemented using Hypernetworks (PHNs) to approximate the Pareto front. This method enables the acquisition of a mapping function from a given preference vector to the solutions on the Pareto front. However, most existing PFL approaches still face two challenges: (a) sampling rays in high-dimensional spaces; (b) failing to cover the entire Pareto Front which has a convex shape. Here, we introduce a novel PFL framework, called as PHN-HVVS, which decomposes the design space into Voronoi grids and deploys a genetic algorithm (GA) for Voronoi grid partitioning within high-dimensional space. We put forward a new loss function, which effectively contributes to more extensive coverage of the resultant Pareto front and maximizes the HV Indicator. Experimental results on multiple MOO machine learning tasks demonstrate that PHN-HVVS outperforms the baselines significantly in generating Pareto front. Also, we illustrate that PHN-HVVS advances the methodologies of several recent problems in the FL field. The code is available at https://github.com/buptcmm/phnhvvs.

Federated learning (FL) is a machine learning paradigm that allows multiple FL participants (FL-PTs) to collaborate on training models without sharing private data. Due to data heterogeneity, negative transfer may occur in the FL training process. This necessitates FL-PT selection based on their data complementarity. In cross-silo FL, organizations that engage in business activities are key sources of FL-PTs. The resulting FL ecosystem has two features: (i) self-interest, and (ii) competition among FL-PTs. This requires the desirable FL-PT selection strategy to simultaneously mitigate the problems of free riders and conflicts of interest among competitors. To this end, we propose an optimal FL collaboration formation strategy -FedEgoists- which ensures that: (1) a FL-PT can benefit from FL if and only if it benefits the FL ecosystem, and (2) a FL-PT will not contribute to its competitors or their supporters. It provides an efficient clustering solution to group FL-PTs into coalitions, ensuring that within each coalition, FL-PTs share the same interest. We theoretically prove that the FL-PT coalitions formed are optimal since no coalitions can collaborate together to improve the utility of any of their members. Extensive experiments on widely adopted benchmark datasets demonstrate the effectiveness of FedEgoists compared to nine state-of-the-art baseline methods, and its ability to establish efficient collaborative networks in cross-silos FL with FL-PTs that engage in business activities.

This article addresses the optimization problem of zeroth-order in a distributed setting, where the gradient information is not available in the edge Internet of Things (IoT) clients. The high communication costs and poorer convergence hinder the use of zeroth-order optimization methods in distributed IoT. This article proposes a communication-efficient Distributed adaptive Zeroth-order optimization method (DaZoo). DaZoo is applied to optimize a class of nonconvex optimization problems, where each client can only access zeroth-order information of local functions. To estimate the global gradient, each client uses a population-based feedback strategy to approximate the first-order gradient, which are then aggregated through a central server. A novel global adaptive optimization scheme is devised for DaZoo, making it with the flexibility to adapt to any landscape without the need for manual parameter tuning. Furthermore, sparsification techniques are incorporated into the local model differences to substantially reduce communication overhead. The theoretical findings suggest that DaZoo can reduce iteration complexity compared to the baselines. Case studies on distributed closed-box attacks and large-scale IoT attack detection demonstrate that DaZoo can outperform state-of-the-art methods.

Dynamic malware detection refers to detecting mal-ware by inferring the run-time trace of malware, i.e., a sequence of API calls. In this paper, we proposed HeteroNet, a novel dynamic malware detection model. The main idea of HeteroNet is that it integrates multiple deep learning models which use heterogeneous dynamic features of malware samples.Specifically, we implement three heterogeneous deep learning based models to learn various features from three representations, namely API name sequence, API resource graph and API call graph, respectively, each of the representation is built from the run-time trace of malware. Meanwhile, several methods such as attention mechanism and graph neural networks are applied in base models, according to the characteristics of API calls. Finally, an ensemble algorithm is used to integrate the outputs of three base models. We trained and evaluated HeteroNet on a dataset of 28,770 samples. The precision of the model on the testing set reached 98.40%, which is 1.20% higher than the best result of baselines. Moreover, HeteroNet is more robust against concept drift than other baselines.

In multiobjective decision making, most knee identification algorithms implicitly assume that the given solutions are well distributed and can provide sufficient information for identifying knee solutions. However, this assumption may fail to hold when the number of objectives is large or when the shape of the Pareto front is complex. To address the above issues, we propose a knee-oriented solution augmentation (KSA) framework that converts the Pareto front into a multimodal auxiliary function whose basins correspond to the knee regions of the Pareto front. The auxiliary function is then approximated using a surrogate and its basins are identified by a peak detection method. Additional solutions are then generated in the detected basins in the objective space and mapped to the decision space with the help of an inverse model. These solutions are evaluated by the original objective functions and added to the given solution set. To assess the quality of the augmented solution set, a measurement is proposed for the verification of knee solutions when the true Pareto front is unknown. The effectiveness of KSA is verified on widely used benchmark problems and successfully applied to a hybrid electric vehicle controller design problem.

In surrogate-assisted multi-/many-objective evolutionary optimization, each solution normally has an approximated value on each objective, resulting in increased difficulties in selecting solutions for expensive objective evaluations due to complicated tradeoff between different objectives and accumulated uncertainty in the approximation of the objective functions. Thus, it is highly challenging to design an efficient model management strategy for surrogate-assisted expensive multi-/many-objective optimization. In this article, a surrogate model is built for each objective function, based on which a set of promising candidate solutions are found. Additionally, a Gaussian process model is constructed to approximate a newly designed performance indicator measuring both convergence and diversity properties of individual solutions. Finally, the solution of the found candidate solutions having the maximum expected improvement in terms of the performance indicator is selected for evaluation using the expensive objective functions. Comparative experiments are conducted on 3-, 5-, and 10-objective DTLZ, WFG, and MaF test functions, as well as two real-world applications. The experimental results show that the proposed method is competitive compared to five state-of-the-art surrogate-assisted evolutionary algorithms for expensive multi-/many-objective optimization.

It is highly desirable to adapt the reference vectors to unknown Pareto fronts (PFs) in decomposition-based evolutionary many-objective optimization. While adapting the reference vectors enhances the diversity of the achieved solutions, it often decelerates the convergence performance. To address this dilemma, we propose to adapt the reference vectors and the scalarizing functions in a coordinated way. On the one hand, the adaptation of the reference vectors is based on a local angle threshold, making the adaptation better tuned to the distribution of the solutions. On the other hand, the weights of the scalarizing functions are adjusted according to the local angle thresholds and the reference vectors’ age, which is calculated by counting the number of generations in which one reference vector has at least one solution assigned to it. Such coordinated adaptation enables the algorithm to achieve a better balance between diversity and convergence, regardless of the shape of the PFs. Experimental studies on MaF, DTLZ, and DPF test suites demonstrate the effectiveness of the proposed algorithm in solving problems with both regular and irregular PFs.

Most reference vector-based decomposition algorithms for solving multiobjective optimization problems may not be well suited for solving problems with irregular Pareto fronts (PFs) because the distribution of predefined reference vectors may not match well with the distribution of the Pareto-optimal solutions. Thus, the adaptation of the reference vectors is an intuitive way for decomposition-based algorithms to deal with irregular PFs. However, most existing methods frequently change the reference vectors based on the activeness of the reference vectors within specific generations, slowing down the convergence of the search process. To address this issue, we propose a new method to learn the distribution of the reference vectors using the growing neural gas (GNG) network to achieve automatic yet stable adaptation. To this end, an improved GNG is designed for learning the topology of the PFs with the solutions generated during a period of the search process as the training data. We use the individuals in the current population as well as those in previous generations to train the GNG to strike a balance between exploration and exploitation. Comparative studies conducted on popular benchmark problems and a real-world hybrid vehicle controller design problem with complex and irregular PFs show that the proposed method is very competitive.

Conventional multiobjective optimization algorithms (MOEAs) with or without preferences are successful in solving multi- and many-objective optimization problems. However, a strong hypothesis underlying their performance is that MOEAs are able to find a representative solution set to cover the entire Pareto-optimal front (PF) and decision makers are able to conveniently and precisely articulate their preference, which is not always easy to fulfill in practice. Accordingly, it is suggested that representative solutions in the naturally interesting regions of the PF rather than the whole PF should be targeted. A large body of research has been proposed to search or identify the knees or knee regions over the past decades. Therefore, this article aims to provide a comprehensive survey of the research on knee-oriented optimization. We start with a discussion of the importance and basic concepts of the knees, followed by a summary of knee-oriented benchmarks and indicators. After that, knee-oriented frameworks and techniques, and real-world applications are presented. Finally, potential challenges are pointed out and a few promising future lines of research are suggested. The survey offers a new perspective to develop MOEAs for solving multi- and many-objective optimization problems.

Acquisition functions for surrogate-assisted many-objective optimization require a delicate balance between convergence and diversity. However, the conflicting nature between many objectives may lead to an imbalance between exploration and exploitation, resulting in a low efficiency in search for a set of optimal solutions that can well balance convergence and diversity. To meet this challenge, we propose an adaptive model management strategy assisted by two sets of reference vectors, one set of adaptive reference vectors accounting for convergence while the other set of fixed reference vectors for diversity. Specifically, we first propose a new acquisition function that calculates an amplified upper confidence bound (AUCB). Two optimization processes are performed in parallel to optimize the acquisition function, each based on one of the two sets of reference vectors. Then, we select one promising candidate solution according to diversity or convergence from the nondominated solutions obtained by the two optimization processes. The experimental results on four suites of test functions as well as six real-world application problems demonstrate the competitive performance of the proposed reference vector-assisted adaptive model management strategy, in comparison with seven state-of-the-art surrogate-assisted evolutionary algorithms (SAEAs).

Neural Architecture Search (NAS) is studied to automatically design the deep neural network structure, freeing people from heavy network design tasks. Traditional NAS based on individual performance evaluation needs to train many networks generated by the search, and compare the performance of the networks according to their accuracy, which is very time-consuming. In this study, we propose to use a two-category comparator based random forest model as a surrogate to estimate the accuracy of the networks. thereby reducing heavy network training process and greatly saving search time. Instead of directly predicting the accuracy of each network, we propose to compare the relative performance between each two networks in our proposed two-category comparator. Furthermore, we implement the modeling process of the surrogate model in the sampling space of the original training data, which further accelerates the search process of the network in the NAS. Experimental results show that our proposed NAS framework can greatly reduce the search time, while the accuracy of the obtained network is comparable to that of other state-of-the art NAS algorithms.

Tyre pattern image retrieval (TPIR) is an important tool in the investigation of criminal activities and traffic accidents. Although content-based image retrieval (CBIR) has been developed for decades with abundant results, the study on TPIR which started in the 1990s has not made much progress. The lack of large standard test datasets is a crucial shortcoming which limits the research in this field. Information presented in this paper is a result of the authors’ literature research on recent academic publications and practical field investigation in the public security and transportation sectors. The state-of-the-art technologies in the field of TPIR are surveyed in detail from two aspects of tyre patterns – their low-level spatial features and high-level semantic features. Existing algorithms are examined and their pros and cons are compared and verified through experimental results. This paper also surveys the available tyre pattern datasets used in all available literature. Finally, with the considerations on technology trends in image retrieval and application requirements in TPIR, the future research directions in this field are laid out.

Various applications of the Internet of Things assisted by deep learning such as autonomous driving and smart furniture have gradually penetrated people’s social life. These applications not only provide people with great convenience but also promote the progress and development of society. However, how to ensure that the important personal privacy information in the big data of the Internet of Things will not be leaked when it is stored and shared on the cloud is a challenging issue. The main challenges include (1) the changes in access rights caused by the flow of manufacturers or company personnel while sharing and (2) the lack of limitation on time and frequency. We propose a data privacy protection scheme based on time and decryption frequency limitation that can be applied in the Internet of Things. Legitimate users can obtain the original data, while users without a homomorphic encryption key can perform operation training on the homomorphic ciphertext. On the one hand, this scheme does not affect the training of the neural network model, on the other hand, it improves the confidentiality of data. Besides that, this scheme introduces a secure two‐party agreement to improve security while generating keys. While revoking, each attribute is specified for the validity period in advance. Once the validity period expires, the attribute will be revoked. By using storage lists and setting tokens to limit the number of user accesses, it effectively solves the problem of data leakage that may be caused by multiple accesses in a long time. The theoretical analysis demonstrates that the proposed scheme can not only ensure safety but also improve efficiency.

For problems with irregular Pareto fronts, only part of the objective space is covered by optimal solutions. Most decomposition based evolutionary many-objective algorithms, however, predefine uniformly distributed weight or reference vectors, making them less suited for problems with irregular Pareto fronts, since many weight or reference vectors will be wasted. To address the above issue, this paper proposes a variant of the reference vector guided evolutionary algorithm by adjusting reference vectors according to the distribution of the solutions in the current population to make sure that most reference vectors are associated with solutions. A secondary selection criterion based on the dominance relationship is adopted in addition to the angle penalized distance based selection so that a sufficient number of solutions can survive and be passed to the next generation. Experiments on 12 irregular test problems with 60 instances show that the proposed algorithm is competitive compared to the state-of-the-art algorithms for solving problems with irregular Pareto fronts.

Texture feature is important in describing tire pattern image which provides useful clue in solving crime cases and traffic accidents. In this paper, we propose a novel texture feature extraction method based on HOG (Histogram of Oriented Gradient) and dominant gradient (DG) in tire pattern images, named HOG-DG. The proposed HOG-DG is not only robust to illumination and scale changes but also is rotation-invariant. In the proposed HOG-DG, HOG features are first computed from circular local cells, and HOG features from an image are concatenated and normalized using the DG to construct the HOG-DG feature. HOG-DG is used to train a support-vector-machine (SVM) classifier for tire pattern classification. Experimental results demonstrate its outstanding performance for tire pattern description.