Unsupervised domain adaptive pose estimation is a fundamental yet challenging task due to the need to transfer from labeled synthetic data to unlabeled real data. Nevertheless, the underlying pose semantics, which are governed by spatial structure, remain largely consistent across domains. This observation motivates the use of vision-language models, which provide domain-invariant representations that align well with high-level semantic concepts. Motivated by this, we propose CLIP2Pose, a novel framework that leverages the semantic robustness of frozen CLIP encoders to facilitate cross-domain generalization. We first introduce a semantic-driven prompt mechanism that encodes structural priors, domain-specific appearance, and instance-level context into the image representation. This guides the model to focus on semantically meaningful and structurally relevant features. Next, we propose a semantic modulation module that adaptively refines visual features by conditioning them on prompt-derived embeddings, enhancing alignment between semantics and visual patterns. To further bridge the modality and domain gaps, we design a directional alignment loss that encourages consistent structural reasoning across both vision and language representations. Extensive experiments on domain adaptive human body and hand pose benchmarks show that CLIP2Pose achieves state-of-the-art performance.

Yixuan Wang, Yue Huang, Hong Qian, Yunzhao Wei, Yifei Ding, Wenkai Wang, Zhi Liu, Zhongjing Huang, Aimin Zhou, Jiajun Guo. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Learner-item cognitive modeling plays a central role in the web-based online intelligent education system by enabling cognitive diagnosis (CD) across diverse online educational scenarios. Although ID embedding remains the mainstream approach in cognitive modeling due to its effectiveness and flexibility, recent advances in language models (LMs) have introduced new possibilities for incorporating rich semantic representations to enhance CD performance. This highlights the need for a comprehensive analysis of how LMs enhance embeddings through semantic integration across mainstream CD tasks. This paper identifies two key challenges in fully leveraging LMs in existing work: Misalignment between the training objectives of LMs and CD models creates a distribution gap in feature spaces; A unified framework is essential for integrating textual embeddings across varied CD tasks while preserving the strengths of existing cognitive modeling paradigms to ensure the robustness of embedding enhancement. To address these challenges, this paper introduces EduEmbed, a unified embedding enhancement framework that leverages fine-tuned LMs to enrich learner-item cognitive modeling across diverse CD tasks. EduEmbed operates in two stages. In the first stage, we fine-tune LMs based on role-specific representations and an interaction diagnoser to bridge the semantic gap of CD models. In the second stage, we employ a textual adapter to extract task-relevant semantics and integrate them with existing modeling paradigms to improve generalization. We evaluate the proposed framework on four CD tasks and computerized adaptive testing (CAT) task, achieving robust performance. Further analysis reveals the impact of semantic information across diverse tasks, offering key insights for future research on the application of LMs in CD for online intelligent education systems.

MOTIVATION: Retrosynthesis plays a crucial role in organic synthesis and drug discovery, focusing on identifying a set of reactants capable of synthesizing a target product molecule. Although the existing approaches have shown promising results, they do not fully exploit 3D conformer information and molecular spatial structure, which can hinder stereochemically consistent and chemically plausible predictions. RESULTS: To tackle this problem, we propose ConfRetro, a Transformer-based template-free method that integrates molecular conformer information and spatial structure. We devise an Atom-align Fusion module to combine 3D positional information at the model input stage, ensuring alignment between atom tokens and corresponding 3D representations. Furthermore, we design a Distance-weighted Attention mechanism to guide self-attention, constraining the receptive field of model and emphasizing chemically relevant atom pairs in 3D space. Experiments conducted on the USPTO-50K and USPTO-FULL datasets demonstrate that ConfRetro significantly outperforms existing template-free approaches, achieving a new state-of-the-art performance. Case studies further highlight its capability to predict accurate and chemically plausible reactants, even for target molecules with intricate structures. Moreover, when plugged into a standard retrosynthetic search, ConfRetro recovers feasible synthetic routes for multiple representative drug molecules (e.g. Camptothecin). AVAILABILITY AND IMPLEMENTATION: ConfRetro is available at https://github.com/Jesse-zjx/ConfRetro. Archival snapshot is available at https://doi.org/10.5281/zenodo.20785018.

Expensive optimization problems pose significant challenges to traditional gradient-free optimization due to their costly evaluation overhead. Surrogate model-assisted evolutionary optimization, which substitutes expensive evaluation functions with surrogate models, can effectively overcome these challenges. Designing an efficient surrogate model is the key issue in model-assisted evolutionary optimization. In recent years, establishing surrogate models through the relationships between solutions has become a promising modeling strategy, following regression and classification models. However, there has been a notable lack of systematic organization or comprehensive summary of relation models, which has impeded the structured development of this burgeoning research area. This paper seeks to address this gap by viewing relations as a perspective to outline the contextual development of the field, defining a robust framework for researching relation models, and reviewing typical strategies within each framework. Finally, it validates the effectiveness of numerous strategies through experiments. The entire collection of strategies will be open-sourced on GitHub1, facilitating greater participation from the research community in this field of study.

As the number and complexity of constraints in constrained multi-objective optimization problems (CMOPs) increase, the performance of existing constrained multi-objective evolutionary algorithms (CMOEAs) declines significantly. A novel idea is to sequentially address each constraint based on priority, effectively reducing the complexity of CMOPs. However, in these algorithms, the constraint-handling priority is determined statically in the initial stage. This may lead to inappropriate determination of constraint-handling priority since accurately estimating the constraint landscape in the initial stage is quite challenging. Moreover, these algorithms tackle constraints separately, neglecting the potential for inter-constraint cooperation and thus compromising their efficiency in constraint handling. Thus, we propose a constrained multi-objective evolutionary algorithm based on dynamic priority and cooperative offspring generation called DPCMOEA. Firstly, the constraint-handling priority is determined dynamically by the estimated inconsistency degree (EID) between the Pareto fronts of the candidate constraints and the current population. Secondly, computational resources are automatically allocated to each constraint according to EID based constraint relationship analysis. Finally, a new offspring generation strategy based on constraint cooperation is designed to enhance the quality of new solutions. Experimental results on six CMOP test suites demonstrate that DPCMOEA outperforms six state-of-the-art algorithms.

In this paper, we introduce a novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms. The goal of model merging is to combine multiple models, each excelling in different tasks, into a single model that outperforms any of the individual source models. However, the effectiveness of conventional model merging methods is constrained by human intuition or domain knowledge. While existing optimization-based model merging methods can automatically search for model merging parameter configurations, they often struggle to find a satisfactory configuration within a limited evaluation budget. To address this challenge, we propose a novel and sample-efficient automated model merging method, named MM-MO. This method leverages multi-objective Bayesian optimization algorithms to autonomously search for great merging configurations across various tasks. In MMMO, we proposed an enhanced acquisition strategy and an auxiliary optimization objective to improve the search process. Our enhanced acquisition strategy integrates a weak-to-strong method to refine the acquisition function, enabling previously evaluated superior configurations to guide the search for new ones. Meanwhile, Fisher information is utilized to further filter these configurations, increasing the possibility of finding high-quality merging configurations. Additionally, we design a sparsity metric as an auxiliary optimization objective, further enhance the models generalization performance across different tasks. We conducted comprehensive experiments with other mainstream model merging methods, demonstrating that the proposed MMMO algorithm is competitive and effective in achieving high-quality model merging.

Multi-objective Bayesian optimization (MOBO) has shown promising performance on various expensive multi-objective optimization problems (EMOPs). However, effectively modeling complex distributions of the Pareto optimal solutions is difficult with limited function evaluations. Existing Pareto set learning algorithms may exhibit considerable instability in such expensive scenarios, leading to significant deviations between the obtained solution set and the Pareto set (PS). In this paper, we propose a novel Composite Diffusion Model based Pareto Set Learning algorithm (CDM-PSL) for expensive MOBO. CDM-PSL includes both unconditional and conditional diffusion model for generating high-quality samples efficiently. Besides, we introduce a weighting method based on information entropy to balance different objectives. This method is integrated with a guiding strategy to appropriately balancing different objectives during the optimization process. Experimental results on both synthetic and real-world problems demonstrates that CDM-PSL attains superior performance compared with state-of-the-art MOBO algorithms.

Yufang Liu, Yao Du, Tao Ji, Jianing Wang, Yang Liu, Yuanbin Wu, Aimin Zhou, Mengdi Zhang, Xunliang Cai. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Jiayi Zeng, Yizhe Feng, Mengliang He, Wenhui Lei, Wei Zhang, Zeming Liu, Xiaoming Shi, Aimin Zhou. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Retrosynthesis prediction is fundamental to drug discovery and chemical synthesis, requiring the identification of reactants that can produce a target molecule. Current templatefree methods struggle to capture the structural invariance inherent in chemical reactions, where substantial molecular scaffolds remain unchanged, leading to unnecessarily large search spaces and reduced prediction accuracy. We introduce C-SMILES, a novel molecular representation that decomposes traditional SMILES into element-token pairs with five special tokens, effectively minimizing editing distance between reactants and products. Building upon this representation, we incorporate a copy-augmented mechanism that dynamically determines whether to generate new tokens or preserve unchanged molecular fragments from the product. Our approach integrates SMILES alignment guidance to enhance attention consistency with ground-truth atom mappings, enabling more chemically coherent predictions. Comprehensive evaluation on USPTO-50K and large-scale USPTO-FULL datasets demonstrates significant improvements: 67.2 % top-1 accuracy on USPTO-50K and 50.8% on USPTO-FULL, with 99.9 % validity in generated molecules. This work establishes a new paradigm for structure-aware molecular generation with direct applications in computational drug discovery.

Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of tasks. However, their proficiency and reliability in the specialized domain of financial data analysis, particularly focusing on data-driven thinking, remain uncertain. To bridge this gap, we introduce FinDABench, a comprehensive benchmark designed to evaluate the financial data analysis capabilities of LLMs within this context. The benchmark comprises 15,200 training instances and 8,900 test instances, all meticulously crafted by human experts. FinDABench assesses LLMs across three dimensions: 1) Core Ability, evaluating the models’ ability to perform financial indicator calculation and corporate sentiment risk assessment; 2) Analytical Ability, determining the models’ ability to quickly comprehend textual information and analyze abnormal financial reports; and 3) Technical Ability, examining the models’ use of technical knowledge to address real-world data analysis challenges involving analysis generation and charts visualization from multiple perspectives. We will release FinDABench, and the evaluation scripts at https://github.com/xxx. FinDABench aims to provide a measure for in-depth analysis of LLM abilities and foster the advancement of LLMs in the field of financial data analysis.

Diffusion models have substantially advanced text-to-image generation, achieving remarkable performance in creating high-quality images from textual prompts. However, they often struggle with accurately generating images representing spatial locations described or implied in the prompts. To address this, we introduce SALT, a training-free method leveraging semantic attention and layout guidance from Large Language Models (LLMs) for text-to-image generation. This method effectively guides both cross-attention and self-attention layers within diffusion models, steering generation toward the direction of high-attention values provided by the layout guidance. During the denoising process of the diffusion model, image features in the latent space are iteratively refined based on the loss function calculated from the desired attention maps. Our approach has been executed on two benchmarks, providing detailed qualitative examples and comprehensive quantitative analyses. Results demonstrate that SALT outperforms existing training-free methods in controlling object layouts and generating attributes.1

Optimization problems are prevalent across various scenarios. Formulating and then solving optimization problems described by natural language often requires highly specialized human expertise, which could block the widespread application of optimization-based decision making. To automate problem formulation and solving, leveraging large language models (LLMs) has emerged as a potential way. However, this kind of approach suffers from the issue of optimization generalization. Namely, the accuracy of most current LLM-based methods and the generality of optimization problem types that they can model are still limited. In this paper, we propose a unified learning-based framework called LLMOPT to boost optimization generalization. Starting from the natural language descriptions of optimization problems and a pre-trained LLM, LLMOPT constructs the introduced five-element formulation as a universal model for learning to define diverse optimization problem types. Then, LLMOPT employs the multi-instruction tuning to enhance both problem formalization and solver code generation accuracy and generality. After that, to prevent hallucinations in LLMs, such as sacrificing solving accuracy to avoid execution errors, the model alignment and self-correction mechanism are adopted in LLMOPT. We evaluate the optimization generalization ability of LLMOPT and compared methods across six real-world datasets covering roughly 20 fields such as health, environment, energy and manufacturing, etc. Extensive experiment results show that LLMOPT is able to model various optimization problem types such as linear/nonlinear programming, mixed integer programming, and combinatorial optimization, and achieves a notable 11.08% average solving accuracy improvement compared with the state-of-the-art methods. The code is available at https://github.com/caigaojiang/LLMOPT.

Black-box optimization aims to find the optima through building a model close to the black-box objective function based on function value evaluation. However, in many real-world tasks, such as the design of molecular formulas and mechanical structures, it is perilous, costly, or even infeasible to evaluate the objective function value of an actively sampled solution. In this situation, optimization can only be conducted via utilizing offline historical data, which yields offline black-box optimization. Different from the traditional goal that is to pursue the optimal solution, this paper emphasizes that the goal of offline optimization is to stably surpass the offline dataset during optimization procedure. Although benchmarks called Design-Bench already exist in this emerging field, it can hardly evaluate the stability of offline optimization and mainly provides real-world offline tasks and the corresponding offline datasets. To this end, this paper proposes benchmarks named SOO-Bench (i.e., Stable Offline Optimization Benchmarks) for offline black-box optimization algorithms, so as to systematically evaluate the stability of surpassing the offline dataset under different data distributions. Along with SOO-Bench, we also propose a stability indicator to measure the degree of stability. Specifically, SOO-Bench includes various real-world offline optimization tasks and offline datasets under different data distributions, involving the fields of satellites, materials science, structural mechanics, and automobile manufacturing. Empirically, baseline and state-of-the-art algorithms are tested and analyzed on SOO-Bench. Hopefully, SOO-Bench is expected to serve as a catalyst for the rapid developments of more novel and stable offline optimization methods. The code is available at \url{https://github.com/zhuyiyi-123/SOO-Bench}.

Causal discovery aims to identify causal relationships from observational data. Recently, optimization-based causal discovery methods have attracted extensive attention in the literature due to their efficiency in handling high-dimensional problems. However, we observe that optimization-based methods often perform well on certain problems but struggle with others. This paper identifies a specific characteristic of causal structural equations that determines the difficulty of identification in causal discovery and, in turn, the performance of optimization-based methods. We conduct an in-depth study of the additive noise model (ANM) and propose to further divide identifiable problems into strongly and weakly identifiable types based on the difficulty of identification. We also provide a sufficient condition to distinguish the two categories. Inspired by these findings, this paper further proposes GENE, a generic method for addressing strongly and weakly identifiable problems in a unified way under the ANM assumption. GENE adopts an order-based search framework that incorporates conditional independence tests into order fitness evaluation, ensuring effectiveness on weakly identifiable problems. In addition, GENE restricts the dimensionality of the effect variables to ensure scale invariance, a property crucial for practical applications. Experiments demonstrate that GENE is uniquely effective in addressing weakly identifiable problems while also remaining competitive with state-of-the-art causal discovery algorithms for strongly identifiable problems.

Pre-trained vision-language models (VLMs) have revolutionized the field of machine learning, demonstrating exceptional performance across a wide range of tasks. However, their robustness remains vulnerable to the spurious-correlation problem. Existing works often involve fine-tuning the model with labeled data or relying on large language models (LLMs) to generate more complex prompts. Although effective to some extent, these methods introduce new challenges, including additional computational costs and dependence on the quality of prompts without fully utilizing the vision modality. To address these limitations, we propose a novel method named ERICT to Enhance model Robustness by Identifying Concept Tokens. ERICT mitigates spurious correlation directly in the inference stage and comprises two key steps: (1) Identify concept tokens capturing invariant features through auxiliary prompts to generate a token-level mask. (2) Apply the mask to the attention weights of the CLS token in the vision encoder to help the model focus on the relevant image region. Extensive experiments show that ERICT significantly improves the overall performance including that of the worst group, and achieves new state-of-the-art results.

Computerized Adaptive Testing (CAT) measures student ability by iteratively selecting informative questions, with core components being the Cognitive Diagnosis Model (CDM) and selection strategy. Current research focuses on optimizing the selection strategy, assuming relatively accurate CDM results. However, existing static CDMs struggle with rapid and accurate diagnosis in the early stage of CAT. To this end, this paper proposes a Fast Adaptive Cognitive Diagnosis (FACD) framework, which incorporates dynamic collaborative and personalized diagnosis modules. Specifically, the collaborative module in FACD uses a dynamic response graph to quickly build student cognitive profiles, while the personalized module leverages each student's response sequence for robust and individualized diagnosis. Extensive experiments on real-world datasets show that, compared with existing static CDMs, FACD not only achieves superior prediction performance across various selection strategies with an improvement between roughly 5%-10% in the early stage of CAT, but also maintains a commendable inference speed.

Computerized Adaptive Testing (CAT) is a testing technique that accurately infers students' proficiency levels using a relatively small number of questions.Most existing CAT systems operate on a question-level adaptive paradigm, which is suitable for practice scenarios.However, in computerized standardized high-stakes examinations such as the GRE and GMAT, this paradigm faces several challenges: (1) the lack of comparability in exam results, (2) high implementation costs due to the reliance on real-time interactions and the financial burden of maintaining CAT testing system, and (3) the difficulty in balancing multiple factors of diagnosis quality, attribute coverage, and question exposure.To address these challenges, we propose a Paper-level Computerized Adaptive Testing (PCAT) and its corresponding evaluation method.PCAT divides an exam into multiple testing stages, where examinees adaptively receive test papers of varying difficulty based on their performance in previous stages.The paper assembly problem in PCAT is solved using a population-based multi-objective optimization (MOO) approach.PCAT offers several advantages: First, the paper-level adaptive mechanism ensures that the questions faced by examinees depend solely on their performance in the earlier stages, maintaining adaptability while enhancing the comparability of results across different examinees.Second, PCAT replaces the selection strategy module in traditional CAT with an assembly module, allowing computationally intensive tasks such as cognitive diagnosis and paper assembly to be completed offline before the exam, eliminating the need for real-time interactions.Additionally, the population-based MOO approach generates a set of high-quality solutions in one run, meeting the demands of frequent administration of standardized high-stakes exams like the GRE and reducing the financial burden of maintaining a large-scale CAT system.Finally, MOO naturally

Large language models (LLMs) have obtained promising results in mathematical reasoning, a foundational human intelligence skill. Most previous studies focus on improving or measuring the performance of LLMs via textual math datasets (e.g., MATH, GSM8K). In this paper, we release a Chinese multimodal math (CMM-Math) dataset, including benchmark and training parts, to evaluate and enhance the mathematical reasoning of LMMs. CMM-Math contains over 28,000 high-quality samples, featuring a variety of problem types (e.g., choice, fill-in-the-blank, analysis) with detailed solutions across 12 grade levels from elementary to high school in China. The problem may contain multiple images, and the visual context may be present in the questions or opinions, which makes this dataset more challenging. Our comprehensive analysis reveals that state-of-the-art LMMs on the CMM-Math face challenges, emphasizing the necessity for further improvements in LMM development. We also propose a Multimodal Mathematical LMM (Math-LMM) to handle the problems with mixed input of multiple images and text segments. The Math-LMM is trained using three stages: foundational pre-training, foundational fine-tuning, and mathematical fine-tuning. The extensive experiments indicate that our model effectively improves math reasoning performance by comparing it with the SOTA LMMs over three multimodal mathematical datasets. We release the datasets on GitHub (https://github.com/ECNU-ICALK/EduChat-Math) and Huggingface (https://huggingface.co/datasets/ecnu-icalk/cmm-math).

Emotion recognition in conversation has attained increasing attention for perceiving user emotion in practical conversational applications. Conversational utterances spoken alternately by different speakers inspire most studies to leverage speaker information based on golden speaker labels. In this work, we challenge the existing paradigm of utilizing available speaker labels with a more realistic scenario, where the speaker identity of each utterance is unknown during inference. We propose Progressive Contrastive Deep Supervision for multimodal emotion recognition in conversation (PCDS), incorporating speaker diarization and emotion recognition into one unified framework. To facilitate joint task learning, we inject speaker and emotion bias into the network progressively via contrastive deep supervision, with the task-irrelevant contrast being the intermediate transition. To obtain explicit speaker dependency, we propose a speaker contrast and clustering module (SCC) to endow the capability of partitioning speakers into groups even when neither speaker label nor number of speakers is known as a priori. Experiments on two ERC benchmarks, including IEMOCAP and MELD demonstrate the effectiveness of the proposed method. We also show that progressive contrastive deep supervision helps reconcile the underlying tension between speaker diarization and emotion recognition. Source code is available from Github[https://github.com/Cross-Innovation-Lab/PCDS/].

A series of surrogate-assisted evolutionary algorithms (SAEAs) have been proposed for expensive multi-objective optimization problems (EMOPs), building cheap surrogate models to replace the expensive real function evaluations. However, the search efficiency of these SAEAs is not yet satisfactory. More efforts are needed to further exploit useful information from the real function evaluations in order to better guide the search process. Facing this challenge, this paper proposes a Hyperbolic Neural Network (HNN) based preselection operator to accelerate the optimization process based on limited evaluated solutions. First, the preselection task is modeled as a multi-label classification problem where solutions are classified into different layers (ordinal categories) through -relaxed objective aggregation. Second, in order to resemble the hierarchical structure of candidate solutions, a hyperbolic neural network is applied to tackle the multi-label classification problem. The reason for using HNN is that hyperbolic spaces more closely resemble hierarchical structures than Euclidean spaces. Moreover, to alleviate the data deficiency issue, a data augmentation strategy is employed for training the HNN. In order to evaluate its performance, the proposed HNN-based preselection operator is embedded into two surrogate-assisted evolutionary algorithms. Experimental results on two benchmark test suites and three real-world problems with up to 11 objectives and 150 decision variables involving seven state-of-the-art algorithms demonstrate the effectiveness of the proposed method.

Large-scale multiobjective optimization problems (LSMOPs), characterized by a substantial number of decision variables, pose significant challenges for many existing evolutionary algorithms. However, the search efficiency of these algorithms is not yet satisfactory. This is mainly because that the search efficiency of these algorithms may deteriorate dramatically since the search space increases exponentially with the number of decision variables. Having this in mind, we proposed a large-Scale multiobjective optimization framework named causal inference-based competitive swarm optimizer (CI-CSO). Specifically, a causal-information-(CI)-based operator is designed for competitive swarm optimizers. First, a causal inference technique named information geometric causal inference (IGCI) is introduced to adequately explore the CI between decision variables and fitness values. To further distinguish the positive or negative impacts of these critical variables on solution quality, a CI processing module is designed, facilitating targeted optimization. To enhance search efficiency, CI-based offspring generator are employed, leveraging the variance of causal effects to dynamically adjust the search step size and sampling range. To evaluate its performance, the proposed CI-based operator is embedded into two multiobjective evolutionary algorithms (MOEAs) (LSTPA and LMOCSO). To demonstrate the effectiveness of the proposed framework, experimental results are presented using the LSMOP test suite and five real-world problems, each involving up to 10 000 decision variables. In addition, six classic algorithms are included for comparison.

Multi-objective optimization problems (MOPs) containing a large number of decision variables, which are also known as large-scale multi-objective optimization problems (LSMOPs), pose great challenges to most existing evolutionary algorithms. This is mainly because that a high dimensional decision space degrades the effectiveness of search operators notably, and balancing convergence and diversity becomes a challenging task. In this paper, we propose a two-population based algorithm for large-scale multi-objective optimization named LSTPA. In the proposed algorithm, solutions are classified in to two subpopulations: a Convergence subPopulation (CP) and a Diversity subPopulation (DP), aiming at convergence and diversity respectively. In order to improve convergence speed, a fitness-aware variation operator (FAVO) is applied to drive DP solutions towards CP. Besides, an adaptive penalty based boundary intersection (APBI) strategy is adopted for environmental selection in order to balance convergence and diversity temporally during different stages of evolution process. Experimental results on benchmark test problems with 100-2000 decision variables demonstrate that the proposed algorithm can achieve the best overall performance compared with several state-of-the-art large-scale multi-objective evolutionary algorithms.

Inverted generational distance (IGD) is an important performance indicator in the field of multiobjective optimization (MOO). Although it has been widely used for decades, applying IGD for fair and accurate performance evaluation remains challenging, with the biggest obstacle being the selection of the reference set. IGD generally represents the distance between the solution set and the Pareto front (PF). Since the real PF is often an infinite set, even if it is known, it is difficult to apply it directly to the calculation of IGD. As a workaround, past research typically samples a finite set, i.e., the reference set, from the PF as an approximation, indirectly used in the IGD calculation. This inevitably introduces a systematic error, which we refer to as discretization error. In this article, we prove an upper bound for the discretization error, demonstrating that if the reference set is sufficiently dense and uniformly distributed on the entire PF, the discretization error will converge to zero. Additionally, we propose a numerical method for the exact calculation of IGD and IGD+. When the analytical expression of the PF is known, this method allows for the direct calculation of IGD and IGD+ using the real PF, thus avoiding discretization error.

To solve dynamic multiobjective optimization problems (DMOPs), the optimization algorithms are required to track the movement of the Pareto set after the environmental changes effectively. Many prediction-based dynamic multiobjective evolutionary algorithms (DMOEAs) have been proposed to address this challenge by utilizing environmental information for population reinitialization. However, when environmental changes are complex, irregular, and severe, the solutions and information during the evolution process often contain noise, making it difficult for prediction-based DMOEAs to accurately predict and reinitialize the population. To address this issue, we propose a novel dynamic multiobjective evolutionary algorithm (DM-DMOEA) which uses a diffusion model-based prediction strategy. In DM-DMOEA, to improve the prediction accuracy, the diffusion model is introduced to extract the relationships of high-quality solutions and reinitialize the population, and a PS estimation method is employed to integrate both historical and new environmental information, providing a set of high-quality solutions for diffusion model training. To speed up the response time, a variational autoencoder (VAE) is used to map the decision space to a latent space, which can reduce the diffusion model size and accelerate the diffusion process. To evaluate the effectiveness of the proposed DM-DMOEA on DMOPs, comprehensive experiments are conducted on several benchmarks and a practical problem. The results show that the DM-DMOEA outperforms other four state-of-the-art DMOEAs in most cases.

In the design of multiobjective particle swarm optimization (MOPSO) algorithms, swarm leaders, i.e., the personal best (pbest) and global best (gbest), are expected to guide the particles toward Pareto-optimal solutions. However, most existing MOPSO algorithms focus on selecting such leaders from the archive of candidate solutions to approximate the Pareto front (PF) that may not yield good approximations of the Pareto set (PS). To address this challenge, this work proposes to predict both pbest and gbest for each particle by explicitly approximating the manifold structure of the PS, following the regularity property of multiobjective optimization problems. Thus, we design a leader prediction-based MOPSO (PPSO) algorithm. In our algorithm, a self-organizing mapping (SOM) method is adopted at each iteration to capture the manifold structure from the current swarm to predict leaders. Specifically, pbest is pinpointed by mapping the particle onto the neuron of SOM, while gbest is estimated by randomly selecting from the neighborhood neurons. In this way, the particles of a swarm in PPSO are guided by the predicted pbest and gbest to approximate the Pareto-optimal solutions. The developed PPSO is empirically verified with several representative algorithms, on several benchmark test instances and real-world problems. Experimental results have demonstrated the advantages of leader prediction for MOPSO over other approaches.

We study the problem of extracting emotions and the causes behind these emotions in conversations. Existing methods either tackle them separately or jointly model them at the coarse-grained level of emotions (fewer emotion categories) and causes (utterance-level causes). In this work, we aim to jointly extract more fine-grained emotions and causes. We construct a fine-grained dataset FG-RECCON, includes 16 fine-grained emotion categories and span-level causes. To further improve the fine-grained extraction performance, we propose to utilize the casual discourse knowledge in a knowledge distillation way. Specifically, the teacher model learns to predict causal connective words between utterances, and then guides the student model in identifying both the fine-grained emotion labels and causal spans. Experimental results demonstrate that our distillation method achieves state-of-the-art performance on both RECCON and FG-RECCON dataset.

Optimizing multiple conflicting black-box objectives simultaneously is a prevalent occurrence in many real-world applications, such as neural architecture search, and machine learning. These problems are known as expensive multi-objective optimization problems (EMOPs) when the function evaluations are computationally or financially costly. Multi-objective Bayesian optimization (MOBO) offers an efficient approach to discovering a set of Pareto optimal solutions. However, the data deficiency issue caused by limited function evaluations has posed a great challenge to current optimization methods. Moreover, most current methods tend to prioritize the quality of candidate solutions, while ignoring the quantity of promising samples. In order to tackle these issues, our paper proposes a novel multi-objective Bayesian optimization algorithm with a data augmentation strategy that provides ample high-quality samples for Pareto set learning (PSL). Specifically, it utilizes Generative Adversarial Networks (GANs) to enrich data and a dominance prediction model to screen out high-quality samples, mitigating the predicament of limited function evaluations in EMOPs. Additionally, we adopt the regularity model to expensive multi-objective Bayesian optimization for PSL. Experimental results on both synthetic and real-world problems demonstrate that our algorithm outperforms several state-of-the-art and classical algorithms.

Cognitive diagnosis assessment is a fundamental and crucial task for student learning. It models the student-exercise interaction, and discovers the students' proficiency levels on each knowledge attribute. In real-world intelligent education systems, generalization and interpretability of cognitive diagnosis methods are of equal importance. However, most existing methods can hardly make the best of both worlds due to the complicated student-exercise interaction. To this end, this paper proposes a symbolic cognitive diagnosis~(SCD) framework to simultaneously enhance generalization and interpretability. The SCD framework incorporates the symbolic tree to explicably represent the complicated student-exercise interaction function, and utilizes gradient-based optimization methods to effectively learn the student and exercise parameters. Meanwhile, the accompanying challenge is that we need to tunnel the discrete symbolic representation and continuous parameter optimization. To address this challenge, we propose to hybridly optimize the representation and parameters in an alternating manner. To fulfill SCD, it alternately learns the symbolic tree by derivative-free genetic programming and learns the student and exercise parameters via gradient-based Adam. The extensive experimental results on various real-world datasets show the superiority of SCD on both generalization and interpretability. The ablation study verifies the efficacy of each ingredient in SCD, and the case study explicitly showcases how the interpretable ability of SCD works.

Differential privacy (DP) has achieved remarkable results in the field of privacy-preserving machine learning. However, existing DP frameworks do not satisfy all the conditions for becoming metrics, which prevents them from deriving better basic private properties and leads to exaggerated values on privacy budgets. We propose Wasserstein differential privacy (WDP), an alternative DP framework to measure the risk of privacy leakage, which satisfies the properties of symmetry and triangle inequality. We show and prove that WDP has 13 excellent properties, which can be theoretical supports for the better performance of WDP than other DP frameworks. In addition, we derive a general privacy accounting method called Wasserstein accountant, which enables WDP to be applied in stochastic gradient descent (SGD) scenarios containing subsampling. Experiments on basic mechanisms, compositions and deep learning show that the privacy budgets obtained by Wasserstein accountant are relatively stable and less influenced by order. Moreover, the overestimation on privacy budgets can be effectively alleviated. The code is available at https://github.com/Hifipsysta/WDP.

Black-box optimization problems, which are common in many real-world applications, require optimization through input-output interactions without access to internal workings. This often leads to significant computational resources being consumed for simulations. Bayesian Optimization (BO) and Surrogate-Assisted Evolutionary Algorithm (SAEA) are two widely used gradient-free optimization techniques employed to address such challenges. Both approaches follow a similar iterative procedure that relies on surrogate models to guide the search process. This paper aims to elucidate the similarities and differences in the utilization of model uncertainty between these two methods, as well as the impact of model inaccuracies on algorithmic performance. A novel model-assisted strategy is introduced, which utilizes unevaluated solutions to generate offspring, leveraging the population-based search capabilities of evolutionary algorithm to enhance the effectiveness of model-assisted optimization. Experimental results demonstrate that the proposed approach outperforms mainstream Bayesian optimization algorithms in terms of accuracy and efficiency.

Causal global optimization (CGO) aims to complete optimization tasks through causal inference. In the high-dimensional CGO problems, traditional causal Bayesian optimization (CBO) methods struggle with the curse of dimensionality attributed to the number of variables in the causal graph, and scale inconsistency among Gaussian Process (GP) models. These issues limit the application of CBO in domains requiring optimization over large causal graphs. To address these limitations, this paper proposes a high-dimensional causal Bayesian optimization (HCBO) algorithm. To address the curse of dimensionality, HCBO introduces a submodularity indicator for variable subsets through the concept of causal intrinsic dimensionality (CID). It then uses the submodular optimization algorithm to find approximations of CID within polynomial sample complexity. Theoretically, we disclose a sufficient condition for CID’s existence. To address the issue of scale inconsistency among GP models, HCBO introduces a scale-normalized scoring function, ensuring stable identification of the optimal GP model corresponding to CID for intervention. Extensive experiments are conducted on high-dimensional synthetic and real-world tasks, i.e., coral ecology and health. The existence of CID is verified across the datasets of all tasks. HCBO achieves state-of-the-art performance in CGO problems and can handle causal graphs at a scale 10 times larger than that manageable by previous CBO methods.

Large Vision-Language Models (LVLMs) have achieved impressive performance, yet research has pointed out a serious issue with object hallucinations within these models.However, there is no clear conclusion as to which part of the model these hallucinations originate from.In this paper, we present an in-depth investigation into the object hallucination problem specifically within the CLIP model, which serves as the backbone for many state-of-the-art visionlanguage systems.We unveil that even in isolation, the CLIP model is prone to object hallucinations, suggesting that the hallucination problem is not solely due to the interaction between vision and language modalities.To address this, we propose a counterfactual data augmentation method by creating negative samples with a variety of hallucination issues.We demonstrate that our method can effectively mitigate object hallucinations for the CLIP model, and we show that the enhanced model can be employed as a visual encoder, effectively alleviating the object hallucination issue in LVLMs. 1

The mainstream paradigm of speech emotion recognition (SER) is identifying the single emotion label of the entire utterance. This line of works neglect the emotion dynamics at fine temporal granularity and mostly fail to leverage linguistic information of speech signal explicitly. In this paper, we propose Emotion Neural Transducer for fine-grained speech emotion recognition with automatic speech recognition (ASR) joint training. We first extend typical neural transducer with emotion joint network to construct emotion lattice for fine-grained SER. Then we propose lattice max pooling on the alignment lattice to facilitate distinguishing emotional and non-emotional frames. To adapt fine-grained SER to transducer inference manner, we further make blank, the special symbol of ASR, serve as underlying emotion indicator as well, yielding Factorized Emotion Neural Transducer. For typical utterance-level SER, our ENT models outperform state-of-the-art methods on IEMOCAP in low word error rate. Experiments on IEMOCAP and the latest speech emotion diarization dataset ZED also demonstrate the superiority of fine-grained emotion modeling. Our code is available at https://github.com/ECNU-Cross-Innovation-Lab/ENT.

Knowledge tracing (KT) plays a crucial role in computer-aided education and intelligent tutoring systems, aiming to assess students' knowledge proficiency by predicting their future performance on new questions based on their past response records. While existing deep learning knowledge tracing (DLKT) methods have significantly improved prediction accuracy and achieved state-of-the-art results, they often suffer from a lack of interpretability. To address this limitation, current approaches have explored incorporating psychological influences to achieve more explainable predictions, but they tend to overlook the potential influences of historical responses. In fact, understanding how models make predictions based on response influences can enhance the transparency and trustworthiness of the knowledge tracing process, presenting an opportunity for a new paradigm of interpretable KT. However, measuring unobservable response influences is challenging. In this paper, we resort to counterfactual reasoning that intervenes in each response to answer what if a student had answered a question incorrectly that he/she actually answered correctly, and vice versa. Based on this, we propose RCKT, a novel response influence-based counterfactual knowledge tracing framework. RCKT generates response influences by comparing prediction outcomes from factual sequences and constructed counterfactual sequences after interventions. Additionally, we introduce maximization and inference techniques to leverage accumulated influences from different past responses, further improving the model's performance and credibility. Extensive experimental results demonstrate that our RCKT method outperforms state-of-the-art knowledge tracing methods on four datasets against six baselines, and provides credible interpretations of response influences. The source code is available at https://github.com/JJCui96IRCKT.

Large-scale dual-stream Vision-Language Pre-training (VLP) models provide an efficient solution for text-image retrieval tasks. Despite this, their performance often falls short of the most current single-stream models, primarily due to limited fine-grained text-image interactions. Recent trends indicate a union of these two types of networks. Some methods adopt a retrieve and rerank strategy, their performance improvements largely hinge on the single-stream encoder during inference. Other approaches utilize knowledge distillation to strengthen either the single-stream encoder or the dual-stream encoder, surpassing their previous capabilities. However, existing distillation techniques typically focus on a single knowledge type, neglecting the richer insights available in the teacher model. To bridge this gap, we introduce a Lightweight and Effective Multi-View Knowledge Distillation approach, named LEMKD, for text-image retrieval. This method effectively utilizes response-based, feature-based and relation-based knowledge, transferring the knowledge from the single-stream encoder to the dual-stream encoder. Our approach is executed on the widely used MS-COCO and Flickr30K datasets. Results demonstrate that LEMKD not only matches the exceptional performance of the most advanced single-stream models but also excels in dual-stream encoder performance amidst the recent integration of single-stream and dual-stream models.

Customer segmentation plays a crucial role in credit risk assessment by dividing users into specific risk levels based on their credit scores. Previous methods fail to comprehensively consider the stability in the segmentation process, resulting in frequent changes and inconsistencies in users' risk levels over time. This increases potential risks to a company. To this end, this paper at first introduces and formalizes the concept of stability regret in the segmentation process. However, evaluating stability is challenging due to its black-box nature and the computational burden posed by vast user data sets. To address these challenges, this paper proposes a large-scale stable customer segmentation approach named LASCA. LASCA consists of two phases: high-quality dataset construction (HDC) and reliable data-driven optimization (RDO). Specifically, HDC utilizes an evolutionary algorithm to collect high-quality binning solutions. RDO subsequently builds a reliable surrogate model to search for the most stable binning solution based on the collected dataset. Extensive experiments conducted on real-world large-scale datasets (up to 0.8 billion) show that LASCA surpasses the state-of-the-art binning methods in finding the most stable binning solution. Notably, HDC greatly enhances data quality by 50%. RDO efficiently discovers more stable binning solutions with a 36% improvement in stability, accelerating the optimization process by 25 times via data-driven evaluation. Currently, LASCA has been successfully deployed in the large-scale credit risk assessment system of Alipay.

Cognitive diagnosis models (CDMs) are designed to learn students' mastery levels using their response logs.CDMs play a fundamental role in online education systems since they significantly influence downstream applications such as teachers' guidance and computerized adaptive testing.Despite the success achieved by existing CDMs, we find that they suffer from a thorny issue that the learned students' mastery levels are too similar.This issue, which we refer to as oversmoothing, could diminish the CDMs' effectiveness in downstream tasks.CDMs comprise two core parts: learning students' mastery levels and assessing mastery levels by fitting the response logs.This paper contends that the oversmoothing issue arises from that existing CDMs seldom utilize response signals on exercises in the learning part but only use them as labels in the assessing part.To this end, this paper proposes an oversmoothing-resistant cognitive diagnosis framework (ORCDF) to enhance existing CDMs by utilizing response signals in the learning part.Specifically, OR-CDF introduces a novel response graph to inherently incorporate response signals as types of edges.Then, ORCDF designs a tailored response-aware graph convolution network (RGC) that effectively captures the crucial response signals within the response graph.Via ORCDF, existing CDMs are enhanced by replacing the input embeddings with the outcome of RGC, allowing for the consideration of response signals on exercises in the learning part.Extensive experiments on real-world datasets show that ORCDF not only *

Cognitive diagnosis is a vital upstream task in intelligent education systems. It models the student-exercise interaction, aiming to infer the students' proficiency levels on each knowledge concept. This paper observes that most existing methods can hardly effectively capture the homogeneous influence due to its inherent complexity. That is to say, although students exhibit similar performance on given exercises, their proficiency levels inferred by these methods vary significantly, resulting in shortcomings in interpretability and efficacy. Given the complexity of homogeneous influence, a hypergraph could be a choice due to its flexibility and capability of modeling high-order similarity which aligns with the nature of homogeneous influence. However, before incorporating hypergraph, one at first needs to address the challenges of distorted homogeneous influence, sparsity of response logs, and over-smoothing. To this end, this paper proposes a hypergraph cognitive diagnosis model (HyperCDM) to address these challenges and effectively capture the homogeneous influence. Specifically, to avoid distortion, HyperCDM employs a divide-and-conquer strategy to learn student, exercise and knowledge representations in their own hypergraphs respectively, and interconnects them via a feature-based interaction function. To construct hypergraphs based on sparse response logs, the auto-encoder is utilized to preprocess response logs and K-means is applied to cluster students. To mitigate over-smoothing, momentum hypergraph convolution networks are designed to partially keep previous representations during the message propagation. Extensive experiments on both offline and online real-world datasets show that HyperCDM achieves state-of-the-art performance in terms of interpretability and capturing homogeneous influence effectively, and is competitive in generalization. The ablation study verifies the efficacy of each component, and the case study explicitly showcases the homogeneous influence captured by HyperCDM.

The fraudulent insurance claim is critical for the insurance industry.Insurance companies or agency platforms aim to confidently estimate the fraud risk of claims by gathering data from various sources.Although more data sources can improve the estimation accuracy, they inevitably lead to increased costs.Therefore, a great challenge of fraud risk verification lies in well balancing these two aspects.To this end, this paper proposes a framework named cost-efficient fraud risk optimization with submodularity (CEROS) to optimize the process of fraud risk verification.CEROS efficiently allocates investigation resources across multiple information sources, balancing the trade-off between accuracy and cost.CEROS consists of two parts that we propose: a submodular set-wise classification model * Equal Contribution.

Continual learning aims to learn new knowledge from a sequence of tasks without forgetting. Recent studies have found that projecting gradients onto the orthogonal direction of task-specific features is effective. However, these methods mainly focus on mitigating catastrophic forgetting by adopting old features to construct projection spaces, neglecting the potential to enhance plasticity and the valuable information contained in previous gradients. To enhance plasticity and effectively utilize the gradients from old tasks, we propose Gradient Projection in Common Null Space (GPCNS), which projects current gradients into the common null space of final gradients under all preceding tasks. Moreover, to integrate both feature and gradient information, we propose a collaborative framework that allows GPCNS to be utilized in conjunction with existing gradient projection methods as a plug-and-play extension that provides gradient information and better plasticity. Experimental evaluations conducted on three benchmarks demonstrate that GPCNS exhibits superior plasticity compared to conventional gradient projection methods. More importantly, GPCNS can effectively improve the backward transfer and average accuracy for existing gradient projection methods when applied as a plugin, which outperforms all the gradient projection methods without increasing learnable parameters and customized objective functions. The code is available at https://github.com/Hifipsysta/GPCNS.

Continual learning emerges as a framework that trains the model on a sequence of tasks without forgetting previously learned knowledge, which has been applied in multiple multimodal scenarios. Recently, prompt-based continual learning has achieved excellent domain adaptability and knowledge transfer through prompt generation. However, existing methods mainly focus on designing the architecture of a generator, neglecting the importance of providing effective guidance for training the generator. To address this issue, we propose Generating Prompts in Latent Space (GPLS), which considers prompts as latent variables to account for the uncertainty of prompt generation and aligns with the fact that prompts are inserted into the hidden layer outputs and exert an implicit influence on classification. GPLS adopts a trainable encoder to encode task and feature information into prompts with reparameterization technique, and provides refined and targeted guidance for the training process through the evidence lower bound (ELBO) related to Mahalanobis distance. Extensive experiments demonstrate that GPLS achieves state-of-the-art performance on various benchmarks. Our code is available at https://github.com/Hifipsysta/GPLS.

In modern telecommunication networks, faults manifest as alarms, generating thousands of events daily. Network operators need an efficient method to identify the root causes of these alarms to mitigate potential losses. This task is challenging due to the increasing scale of telecommunication networks and the interconnected nature of devices, where one fault can trigger a cascade of alarms across multiple devices within a topological network. Recent years have seen a growing focus on causal approaches to addressing this problem, emphasizing the importance of learning a Granger causal graph from topological event sequences. Such causal graphs delineate the relations among alarms and can significantly aid engineers in identifying and rectifying faults. However, existing methods either ignore the topological relationships among devices or suffer from relatively low scalability and efficiency, failing to deliver high-quality responses in a timely manner. To this end, this paper proposes $S^2GCSL$, a simple yet scalable Granger causal structural learning approach for topological event sequences. $S^2GCSL$ utilizes a linear kernel to model activation interactions among various event types within a topological network, and employs gradient descent to efficiently optimize the likelihood function. Notably, it can seamlessly incorporate expert knowledge as constraints within the optimization process, which enhances the interpretability of the outcomes. Extensive experimental results on both large-scale synthetic and real-world problems verify the scalability and efficacy of $S^2GCSL$.

Cognitive diagnosis aims to gauge students' mastery levels based on their response logs. Serving as a pivotal module in web-based online intelligent education systems (WOIESs), it plays an upstream and fundamental role in downstream tasks like learning item recommendation and computerized adaptive testing. WOIESs are open learning environment where numerous new students constantly register and complete exercises. In WOIESs, efficient cognitive diagnosis is crucial to fast feedback and accelerating student learning. However, the existing cognitive diagnosis methods always employ intrinsically transductive student-specific embeddings, which become slow and costly due to retraining when dealing with new students who are unseen during training. To this end, this paper proposes an inductive cognitive diagnosis model (ICDM) for fast new students' mastery levels inference in WOIESs. Specifically, in ICDM, we propose a novel student-centered graph (SCG). Rather than inferring mastery levels through updating student-specific embedding, we derive the inductive mastery levels as the aggregated outcomes of students' neighbors in SCG. Namely, SCG enables to shift the task from finding the most suitable student-specific embedding that fits the response logs to finding the most suitable representations for different node types in SCG, and the latter is more efficient since it no longer requires retraining. To obtain this representation, ICDM consists of a construction-aggregation-generation-transformation process to learn the final representation of students, exercises and concepts. Extensive experiments across real-world datasets show that, compared with the existing cognitive diagnosis methods that are always transductive, ICDM is much more faster while maintains the competitive inference performance for new students.

Abstract Due to the scattered, unordered, and unstructured nature of point clouds, it is challenging to extract local features. Existing methods tend to design redundant and less‐discriminative spatial feature extraction methods in the encoder, while neglecting the utilisation of uneven distribution in the decoder. In this paper, the authors fully exploit the characteristics of the imbalanced distribution in point clouds and design our Position‐aware Encoder (PAE) module and Position‐aware Decoder (PAD) module. In the PAE module, the authors extract position relationships utilising both Cartesian coordinate system and polar coordinate system to enhance the distinction of features. In the PAD module, the authors recognise the inherent positional disparities between each point and its corresponding upsampled point, utilising these distinctions to enrich features and mitigate information loss. The authors conduct extensive experiments and compare the proposed DPANet with existing methods on two benchmarks S3DIS and Semantic3D. The experimental results demonstrate that the method outperforms the state‐of‐the‐art approaches.

Change address identification is one of the difficulties in bitcoin address clustering as an emerging social computing problem. Most of the current-related research only applies to certain specific types of transactions and faces the problems of low recognition rate and high false positive rate. We innovatively propose a clustering method based on multiconditional recognition of one-time change addresses and conduct experiments with on-chain bitcoin transaction data. The results show that the proposed method identifies at least 12.3% more one-time change addresses than other heuristics. On top of the multi-input heuristic clustering method, the proposed method also improves the address clustering performance by 5.7%, achieves optimal recognition results compared with similar methods, and significantly reduces the false positive rate of recognition results. This work provides the technical basis for antimoney laundering efforts based on entity identification. Code and data could be accessed from https://github.com/ECNU-Cross-Innovation-Lab/BitcoinAddressClustering.

Recent years have witnessed the repaid progress in developing and applying multiobjective evolutionary algorithms (MOEAs). However, as a major component of an MOEA, the offspring generator has been largely overlooked, lacking a principle to design generators. This paper addresses this issue by introducing an offspring generation paradigm, called regularity evolution (RE), for MOEAs. RE assumes that a solution consists of two parts: a structure vector and a perturbation vector. The former represents the manifold structure learned from the population, in accordance with the regularity property of multiobjective optimization problems, while the latter represents the noise or uncertainty that can be embedded in the learned manifold structure. With the RE paradigm, we can explain and improve some existing generation operators, e.g., the regularity model based generators, and furthermore, design new generators by proposing alternative ways to construct structure or perturbation vectors. The systematic studies with comparisons on popular generation operators and newly developed MOEAs indicate that the RE paradigm has significant superiority in offspring generation for multiobjective optimization.

Ensemble learning methods have been widely used in machine learning in recent years due to their high predictive performance. With the development of genetic programming-based symbolic regression methods, many papers begin to choose a popular ensemble learning method, random forests, as the baseline competitor. Instead of considering them as competitors, an alternative idea might be to consider symbolic regression as an enhancement technique for random forest. Genetic programming-based symbolic regression methods which fit a smooth function are complementary to the piecewise nature of decision trees, as the smooth variation is common in regression problems. In this article, we propose to form an ensemble model with symbolic regression-based decision trees to address this issue. Furthermore, we design a guided mutation operator to speed up the search on high-dimensional problems, a multi-fidelity evaluation strategy to reduce the computational cost and an ensemble selection mechanism to improve predictive performance. Finally, experimental results on a regression benchmark with 120 datasets show that the proposed ensemble model outperforms 25 existing symbolic regression and ensemble learning methods. Moreover, the proposed method can provide notable insights on an XGBoost hyperparameter performance prediction task, which is an important application area of ensemble learning methods.

In many scenarios of black-box optimization, evaluating the objective function values of solutions is expensive, while comparing a pair of solutions is relatively cheap, which yields the dueling black-box optimization. The side effect of dueling optimization is that it doubles the dimension of solution space and exacerbates the dimensionality scalability issue of black-box optimization, e.g., Bayesian optimization. To address this issue, the existing dueling optimization methods fix one solution when dueling throughout the optimization process, but it may reduce their efficacy. Fortunately, it has been observed that, in recommendation systems, the dueling results are mainly determined by the latent human preferences. In this paper, we abstract this phenomenon as the preferential intrinsic dimension and inject it into the dueling Bayesian optimization, resulting in the preferential embedding dueling Bayesian optimization (PE-DBO). PE-DBO decouples optimization and pairwise comparison via the preferential embedding matrix. Optimization is performed in the preferential intrinsic subspace with much lower dimensionality, while pairwise comparison is completed in the original dueling solution space. Theoretically, we disclose that the preference function can be approximately preserved in the lower-dimensional preferential intrinsic subspace. Experiment results verify that, on molecule discovery and web page recommendation dueling optimization tasks, the preferential intrinsic dimension exists and PE-DBO is superior in scalability compared with that of the state-of-the-art (SOTA) methods.

Dynamic Facial Expression Recognition (DFER) is a rapidly developing field that focuses on recognizing facial expressions in video format. Previous research has considered non-target frames as noisy frames, but we propose that it should be treated as a weakly supervised problem. We also identify the imbalance of short- and long-term temporal relationships in DFER. Therefore, we introduce the Multi-3D Dynamic Facial Expression Learning (M3DFEL) framework, which utilizes Multi-Instance Learning (MIL) to handle inexact labels. M3DFEL generates 3D-instances to model the strong short-term temporal relationship and utilizes 3DCNNs for feature extraction. The Dynamic Long-term Instance Aggregation Module (DLIAM) is then utilized to learn the long-term temporal relationships and dynamically aggregate the instances. Our experiments on DFEW and FERV39K datasets show that M3DFEL outperforms existing state-of-the-art approaches with a vanilla R3D18 backbone. The source code is available at https://github.com/faceeyes/M3DFEL.

Cognitive diagnosis is vital for intelligent education to determine students’ knowledge mastery levels from their response logs. The Q-matrix, representing the relationships between exercises and knowledge attributes, improves the interpretability of cognitive diagnosis models. However, completing the Q-matrix poses an expensive and challenging task due to the fine-grained division of knowledge attributes. Moreover, a manually sparse Q-matrix can also compromise the accuracy and interpretability of deducing students’ mastery levels, especially for infrequently observed or unseen knowledge attributes. To address this issue, this paper proposes a Q-augmented Causal Cognitive Diagnosis Model (QCCDM) for student learning. Specifically, QCCDM incorporates the structure causal model (SCM) to capture the causality between students’ mastery levels on different attributes, which enables to infer their proficiency on rarely observed knowledge attributes with better accuracy and interpretability. Notably, with SCM, one can guide students on how to realize their self-improvement through intervention. Furthermore, we propose to augment the Q-matrix in QCCDM, which uses the manual Q-matrix as a prior to deduce the relationships between exercises and explicit as well as latent knowledge attributes, resulting in a complete and comprehensive assessment of students’ abilities. We assess the efficacy of Q-augmentation across the widely-used Q-based cognitive diagnosis models and conduct the ablation study. The extensive experimental results on real-world datasets show that QCCDM outperforms the compared methods in terms of both accuracy and interpretability.

Offline optimization aims to elaborately construct a solution that optimizes a black-box function with only access to the offline dataset. A typical manner of constructing the solution is to train a surrogate model of the black-box function on the offline dataset and optimize the solution guided by the surrogate model. However, this manner often encounters a fundamental challenge that the surrogate model could erroneously estimate out-of-distribution (OOD) solutions. Therefore, the optimizer would be misled to produce inferior solutions for online applications, i.e., degradation of performance. To this end, this paper formalizes the risk of degradation for OOD solutions and proposes an accumulative risk controlled offline optimization (ARCOO) method. Specifically, ARCOO learns a surrogate model in conjunction with an energy model. The energy model characterizes the risk of degradation by learning on high-risk solutions and low-risk ones contrastively. In the optimization procedure, the behavior of the optimizer in each step is controlled by a risk suppression factor calculated via the energy model, which leads to the controllable accumulative risk. Theoretically, we justify the efficacy of energy for accumulative risk control. Extensive experiments on offline optimization tasks show that ARCOO surpasses state-of-the-art methods in both degradation-resistance and optimality of the output solution.

Due to computationally and/or financially costly evaluation, tackling expensive multi-objective optimization problems is quite challenging for evolutionary algorithms. One popular approach to these problems is building cheap surrogate models to replace the expensive real function evaluations. To this end, various kinds of surrogate-assisted evolutionary algorithms (SAEAs) have been proposed, building surrogate models which predict the fitness values, classifications, or relation of the candidate solutions. However, off-spring generation, despite its important role in evolutionary optimization, has not received enough attention in these SAEAs. In this paper, a regularity model based framework, namely RM-SAEA, is proposed for better offspring generation in expensive multi-objective optimization. To be specific, RM-SAEA is featured with a heterogeneous offspring generation module, which is composed of a regularity model and a general genetic operator. Moreover, in order to alleviate the data deficiency issue in the expensive optimization scenario, a data augmentation strategy is employed while training the regularity model. Finally, two representative SAEAs are embedded into RM-SAEA in order to instantiate the proposed framework. Experimental results on benchmark multi-objective problems with up to 10 objectives demonstrate that RM-SAEA achieves the best overall performance compared with 6 state-of-the-art algorithms.

Fueled by recent advances of self-supervised models, pre-trained speech representations proved effective for the downstream speech emotion recognition (SER) task. Most prior works mainly focus on exploiting pre-trained representations and just adopt a linear head on top of the pre-trained model, neglecting the design of the downstream network. In this paper, we propose a temporal shift module to mingle channel-wise information without introducing any parameter or FLOP. With the temporal shift module, three designed baseline building blocks evolve into corresponding shift variants, i.e. ShiftCNN, ShiftLSTM, and Shiftformer. Moreover, to balance the trade-off between mingling and misalignment, we propose two technical strategies, placement of shift and proportion of shift. The family of temporal shift models all outperforms the state-of-the-art methods on the benchmark IEMOCAP dataset under both finetuning and feature extraction settings. Our code is available at https://github.com/ECNU-Cross-Innovation-Lab/ShiftSER.

In recent years, it is a difficult issue to integrate the deep cross-fertilization and interpretable cognitive modeling methods from the basic theory of emotional psychology with deep learning and other algorithms. To address this problem, a cognitive model that integrates the VGG-facial action coding system (FACS)-OCC model based on fer2013 expression features and the OCC-pleasure-arousal-dominance (PAD)-openness, conscientiousness, extraversion, agreeableness, and neuroticism (OCEAN) fusion of the basic theory of emotional psychology, namely, a computational affection-based OCC-PAD-OCEAN federation cognitive modeling (OPO-FCM), is constructed. By constructing this model and performing formal proof algorithms, it is shown that the OPO-FCM can acquire expression features in video streams, complete the acquisition of expression features in videos by training a deep neural network, map expressions to the PAD emotion space through the established expression–basic emotions–emotion space mapping relationship, and finally complete the mapping of the average emotion over a period time. The information of personality space is obtained through it. Finally, the experimental simulation of the model is conducted, and the results show that the average accuracy of the valid tested personalities is 79.56%. This article takes the knowledge-driven approach of emotional psychology as a starting point and combines deep learning techniques to construct interpretable cognitive models, thus providing new ideas for future cross-innovation between computer technology and psychology theory.

The main target of multiobjective evolutionary algorithms (MOEAs) is to find a set of evenly distributed nondominated solutions that approximate the Pareto front (PF) of a multiobjective optimization problem (MOP). This means that the approximated set should be as close to the PF as possible, and as diverse as possible. The former is usually called a convergence criterion and the latter is called a diversity criterion. A variety of strategies have been proposed to meet the two criteria. However, as far as the diversity criterion is concerned, it is still a challenge to achieve an evenly distributed approximation set with different sizes for a problem with a complicated PF shape. To deal with this challenge, we propose a local subset selection (LSS) -based environmental selection for evolutionary multiobjective optimization in this article. LSS considers the environmental selection as a subset selection problem by choosing promising solutions from the combination of the parent and offspring populations. In LSS, a potential energy function is utilized as the objective function, which provides a heavy selection pressure on diversity as well as has low computational complexity. Furthermore, to balance search efficiency and quality, a local search strategy is used in LSS to make full use of objective information for acceleration. The proposed LSS strategy is embedded into some state-of-the-art Pareto-domination-based MOEAs, and the experimental results suggest that LSS can produce shape-invariant and evenly distributed nondominated sets with different population sizes.

Knowledge tracing, the goal of which is predicting students’ future performance given their past question response sequences to trace their knowledge states, is pivotal for computer-aided education and intelligent tutoring systems. Although many technical efforts have been devoted to modeling students based on their question-response sequences, fine-grained interaction modeling between question-response pairs within each sequence is underexplored. This causes question-response representations less contextualized and further limits student modeling. To address this issue, we first conduct a data analysis and reveal the existence of complex cross effects between different question-response pairs within a sequence. Consequently, we propose MRT-KT, a multi-relational transformer for knowledge tracing, to enable fine-grained interaction modeling between question-response pairs. It introduces a novel relation encoding scheme based on knowledge concepts and student performance. Comprehensive experimental results show that MRT-KT outperforms state-of-the-art knowledge tracing methods on four widely-used datasets, validating the effectiveness of considering fine-grained interaction for knowledge tracing.

The estimation of distribution algorithm (EDA) is an efficient heuristic method for handling black-box optimization problems since the ability for global population distribution modeling and gradient-free searching. However, the trial and error search mechanism relies on a large number of function evaluations, which is a considerable challenge under expensive black-box problems. Therefore, this article presents a surrogate assisted EDA with multi-acquisition functions. Firstly, a variable-width histogram is used as the global distribution model that focuses on promising areas. Next, the evaluated-free local search method improves the quality of new generation solutions. Fi-nally, model management with multiple acquisitions maintains global and local exploration preferences. Several commonly used benchmark functions with 20 and 50 dimensions are adopted to evaluate the proposed algorithm compared with several state-of-the-art surrogate assisted evaluation algorithms (SAEAs) and Bayesian optimization method. In addition, a rover trajectories optimizing problem is used to verify the ability to solve complex problems. The experimental results demonstrate the superiority of the proposed algorithm over these comparison algorithms.

Fusion technology is crucial for multimodal sentiment analysis. Recent attention-based fusion methods demonstrate high performance and strong robustness. However, these approaches ignore the difference in information density among the three modalities, i.e., visual and audio have low-level signal features and conversely text has high-level semantic features. To this end, we propose a non-homogeneous fusion network (NHFNet) to achieve multimodal information interaction. Specifically, a fusion module with attention aggregation is designed to handle the fusion of visual and audio modalities to enhance them to high-level semantic features. Then, cross-modal attention is used to achieve information reinforcement of text modality and audio-visual fusion. NHFNet compensates for the differences in information density of different modalities enabling their fair interaction. To verify the effectiveness of the proposed method, we set up the aligned and unaligned experiments on the CMU-MOSEI dataset, respectively. The experimental results show that the proposed method outperforms the state-of-the-art. Codes are available at https://github.com/skeletonNN/NHFNet.

Teaching dimension (TD) is a fundamental theoretical property for understanding machine teaching algorithms. It measures the sample complexity of teaching a target hypothesis to a learner. The TD of linear learners has been studied extensively, whereas the results of teaching non-linear learners are rare. A recent result investigates the TD of polynomial and Gaussian kernel learners. Unfortunately, the theoretical bounds therein show that the TD is high when teaching those non-linear learners. Inspired by the fact that regularization can reduce the learning complexity in machine learning, a natural question is whether the similar fact happens in machine teaching. To answer this essential question, this paper proposes a unified theoretical framework termed STARKE to analyze the TD of regularized kernel learners. On the basis of STARKE, we derive a generic result of any type of kernels. Furthermore, we disclose that the TD of regularized linear and regularized polynomial kernel learners can be strictly reduced. For regularized Gaussian kernel learners, we reveal that, although their TD is infinite, their epsilon-approximate TD can be exponentially reduced compared with that of the unregularized learners. The extensive experimental results of teaching the optimization-based learners verify the theoretical findings.

Abstract Skeleton‐based action recognition has been extensively studied in recent years and applied in virtual reality, detection systems and other cases with strong requirements for low cost as well as high accuracy, but most of the existing methods mainly focus on complex architecture of deep neural networks without considering computation efficiency. To balance accuracy and computation cost well, this paper proposes a simple and efficient attention‐enhanced semantic‐guided graph convolutional network (AeS‐GCN) for skeleton‐based action recognition. Firstly, we fuse semantics of joint type and frame index and dynamics together as representation of skeleton. Then, we use spatial attention block (SAB) to explore important features in spatial structure, in which adaptive GCN layer is adopted to adaptively model skeleton topology structure. Next, we use temporal attention block (TAB) to extract latent temporal information. The model proposed is a lightweight network and achieves the state‐of‐the‐art performance on mainstream datasets with less parameters and less computational complexity.

Expensive multiobjective optimization problems pose great challenges to evolutionary algorithms due to their costly evaluation. Building cheap surrogate models to replace the expensive real models has been proved to be a practical way to reduce the number of costly evaluations. Supervised learning techniques from the community of machine learning have been widely applied to build either regressors, which approximate the fitness values of candidate solutions, or classifiers, which estimate the categories of candidate solutions. Considering the characteristics of the data produced in optimization, this article proposes a new surrogate model, called a relation model, for evolutionary multiobjective optimization. Instead of estimating the qualities of candidate solutions directly, the relation model tries to estimate the relationship between a pair of solutions$\langle \mathbf {x}, \mathbf {y}\rangle $, i.e.,$\mathbf {x}$dominates$\mathbf {y}$,$\mathbf {x}$is dominated by$\mathbf {y}$, or$\mathbf {x}$is nondominated with$\mathbf {y}$in the case of multiobjective optimization. To implement this idea, first a balanced training set is prepared, then a classifier is built based on the training data set to learn the relationship, and finally, the classifier with a voting-scoring strategy is applied to estimate the relationship between the candidate solutions and parent solutions. By this way, the promising candidate solutions are recognized and evaluated by the real models. The new approach is applied to three well-known benchmark suites and two real-world applications, and the experimental results suggest that the proposed method outperforms some state-of-the-art methods based on regression and classification models on the given instances.

Random forest (RF) is a type of ensemble-based machine learning method that has been applied to a variety of machine learning tasks in recent years. This article proposes an evolutionary approach to generate an oblique RF for regression problems. More specifically, our method induces an oblique RF by transforming the original feature space to a new feature space through the evolutionary feature construction method. To speed up the searching process, the proposed method evaluates each set of features based on a decision tree (DT) rather than an RF. In order to obtain an RF, we archive top-performing features and corresponding trees during the search. In this way, both the features and the forest can be constructed simultaneously in a single run. The proposed evolutionary forest is applied to 117 benchmark problems with different characteristics and compared with some state-of-the-art regression methods, including several variants of the RF and gradient boosted DTs (GBDTs). The experimental results suggest that the proposed method outperforms the existing RF and GBDT methods.

Preselection is an important strategy to improve evolutionary algorithms' performance by filtering out unpromising solutions before fitness evaluations. This paper introduces a pre-selection strategy based on an approximated Pareto domination relationship for multiobjective evolutionary optimization. For each objective, a binary relation between each pair of solutions is constructed based on the current population, and a binary classifier is built based on the binary relation pairs. In this way, an approximated Pareto domination relationship can be defined. When new trial solutions are generated, the approximated Pareto domination is used to select promising solutions, which shall be evaluated by the real objective functions. The new preselection is integrated into two algorithms. The experimental results on two benchmark test suites suggest that the algorithms with preselection outperform their original ones.

Symbolic regression has become a hot topic in recent years due to the surging demand for interpretable machine learning methods. Traditionally, symbolic regression problems are mainly solved by genetic algorithms. Nonetheless, with the development of deep learning, reinforcement learning based symbolic regression methods have received attention gradually. Unfortunately, hardly any of those reinforcement learning based methods have been proven effectively to solve real world regression problems as genetic algorithm based methods. In this paper, we find a general reinforcement learning based symbolic regression method is difficult to solve real world problems since it is hard to balance between exploration and exploitation. To deal with this problem, we propose a hybrid method to use both genetic algorithm and reinforcement learning for solving symbolic regression problems. By doing so, we can combine the advantages of reinforcement learning and genetic algorithm and achieve better performance than using them alone. To validate the effectiveness of the proposed method, we apply the proposed method to ten benchmark datasets. The experimental results show that the proposed method achieves competitive performance compared with several well-known symbolic regression methods on those datasets.

With the continuous development and popularity of depth cameras, skeleton-based human action recognition has attracted people's wide attention. Graph Convolutional Network (GCN) has achieved remarkable performance. However, the existing methods do not better consider the semantic characteristics, which can help to express the current concept and scene information. Semantic information can also help with better granularity classification. In addition, most of the existing models require a lot of computation. What's more, adaptive GCN can automatically learn the graph structure and consider the connections between joints. In this paper, we propose a relatively less computationally intensive model, which combines semantic and adaptive graph network (SAGN) for skeleton-based human action recognition. Specifically, we mainly combine the dynamic characteristics and bone information to extract the data, taking the correlation between semantics into the model. In the training process, SAGN includes an adaptive network so that we can make attention mechanism more flexible. We design the Convolutional Neural Network (CNN) for feature extraction on the time dimension. The experimental results show that SAGN achieves the state-of-the-art performance on NTU-RGB+D 60 and NTU-RGB+D 120 datasets. SAGN can promote the study of skeleton-based human action recognition. The source code is available at https://github.com/skeletonNN/SAGN.

Cooperative co-evolution (CC) is an efficient and practical evolutionary framework for solving large-scale optimization problems. The performance of CC is affected by the variable decomposition. An accurate variable decomposition can help to improve the performance of CC on solving an optimization problem. The variable grouping methods usually spend many computational resources obtaining an accurate variable decomposition. To reduce the computational cost on the decomposition, we propose an efficient recursive differential grouping (ERDG) method in this article. By exploiting the historical information on examining the interrelationship between the variables of an optimization problem, ERDG is able to avoid examining some interrelationship and spend much less computation than other recursive differential grouping methods. Our experimental results and analysis suggest that ERDG is a competitive method for decomposing large-scale continuous problems and improves the performance of CC for solving the large-scale optimization problems.

Designing neural networks often requires a large number of artificial intelligence experts. However, such manual processes are time-consuming and require numerous resources. In this paper, we try to search neural network structures automatically for the image classification task. Moreover, considering the huge computational cost of neural architecture search (NAS), we attempt to apply a classification surrogate model based multi-objective evolutionary algorithm to search neural network architectures (CSMEA-Net). The algorithm combines two objectives, i.e., minimizing the validation error and the computational complexity measured by the number of floating-point operations (FLOPs) to achieve Pareto Optimality. In addition, we improve the components of the cell-based search space. The performance of network architectures discovered by our method is evaluated on CIFAR-10 and CIFAR-100 datasets. The experimental results show that the proposed approach can find a higher-performance neural network architecture compared with both hand-crafted as well as automatically-designed networks.

Evaluation metrics play an important role in accessing the performance of a regression method. In practice, these multiple evaluation metrics can be used in two ways. The first way defines a loss function by aggregating multiple metrics, while the second way defines a multiobjective loss function by considering each metric as an objective function. In this paper, we propose a new way to use multiple evaluation metrics, which is different from the aggregating method and the mutliobjective method. Our method is based on genetic programming. The idea is to randomly use one metric in each iteration of the selection operator. Therefore, multiple metrics can be used alternatively in the running process. To validate the effectiveness of our new approach, we conduct experiments on ten benchmark datasets. The experimental results show that the new approach can improve the population diversity, and can achieve the performance better than or similar to that of the traditional symbolic regression algorithms.

Evolutionary algorithms (EAs) are a kind of population-based heuristic optimization method by using trial-and-error. Therefore, the search efficiency is a major concern in both of the algorithm design and applications. The preselection, which estimates the quality of candidate solutions and discards unpromising ones before fitness evaluation, is a widely used component for reducing the number of fitness evaluations in EAs. The surrogate models, such as regression and classification, are usually applied for quality estimation. In some EA frameworks, the relationship between a pair of solutions helps to distinguish “good” and “bad” solutions. In such cases, it is not necessary to estimate the specific quality of each candidate solution but the binary relationship of a pair of solutions. Following this idea, this article proposes a new preselection strategy, called relationship classification-based preselection (RCPS). In RCPS, a classification model is built to learn the relationship between a pair of solutions based on a given training data set, and promising candidate solutions are prescreened by this relation. The mechanism of RCPS is visualized and analyzed. The advantages of RCPS over traditional surrogate model-based preselection strategies are illustrated through a comprehensive empirical study. The experimental results suggest that on two sets of test suits, RCPS outperforms the comparison preselection strategies. To achieve a same accuracy, an EA with RCPS needs a smaller number of fitness evaluations than the one without RCPS.

As one of the classical problems in the economic market, the newsvendor problem aims to make maximal profit by determining the optimal order quantity of products. However, the previous newsvendor models assume that the selling price of a product is a predefined constant and only regard the order quantity as a decision variable, which may result in an unreasonable investment decision. In this article, a new newsvendor model is first proposed, which involves of both order quantity and selling price as decision variables. In this way, the newsvendor problem is reformulated as a mixed-variable nonlinear programming problem, rather than an integer linear programming problem as in previous investigations. In order to solve the mixed-variable newsvendor problem, a histogram model-based estimation of distribution algorithm (EDA) called EDAmvnis developed, in which an adaptive-width histogram model is used to deal with the continuous variables and a learning-based histogram model is applied to deal with the discrete variables. The performance of EDAmvn was assessed on a test suite with eight representative instances generated by the orthogonal experiment design method and a real-world instance generated from real market data of Alibaba. The experimental results show that, EDAmvnoutperforms not only the state-of-the-art mixed-variable evolutionary algorithms, but also a commercial software, i.e., Lingo.

In evolutionary optimization, the preselection is an efficient operator to improve the search efficiency, which aims to filter unpromising candidate solutions before fitness evaluation. Most existing preselection operators rely on fitness values, surrogate models, or classification models. Basically, the classification based preselection regards the preselection as a classification procedure, i.e., differentiating promising and unpromising candidate solutions. However, the difference between promising and unpromising classes becomes fuzzy as the running process goes on, as all the left solutions are likely to be promising ones. Facing this challenge, this paper proposes a fuzzy classification based preselection (FCPS) scheme, which utilizes the membership function to measure the quality of candidate solutions. The proposed FCPS scheme is applied to two state-of-the-art evolutionary algorithms on a test suite. The experimental results show the potential of FCPS on improving algorithm performance.

Pan-sharpening aims to integrate the spatial details of high resolution panchromatic image (PAN) with the spectral information of the corresponding low resolution multispectral image (MS) to produce high resolution multispectral image, which is a challenge real-world task. In this paper, we propose a novel pan-sharpening method, called multiobjective Intensity-Hue-Saturation (MIHS) transformation, which combines adaptive Intensity-Hue-Saturation transformation and evolutionary multi-objective optimization techniques for remote sensing images. The basic idea is to convert the pan-sharpening problem into a mutiobjective optimization problem and deal with it by an multiobjective evolutionary algorithm. The proposed method is applied to two Quick-bird images. Experimental results demonstrate that the proposed method does markedly improve the pan-sharpening performance compared to the state-of-the-art methods and an evolutionary method with single objective in terms of both subjective visual effects and objective quality metrics.

Portfolio optimization is an important financial problem, which involves an optimal allocation of finite capital to a series of assets to achieve an acceptable trade-off between profit and risk in a given investment period. In this paper, an extended Markowitz's mean-variance portfolio optimization model, which is converted as a constrained multi-objective problem, is studied. Since this model involves both discrete and continuous variables, a multi-dimensional mapping coding scheme (MDM) has been adopted to convert discrete variables to continuous ones. Although the basic MDM is effective for dealing with the constrained multi-objective portfolio optimization problems, it sometimes prefers to choose some balanced investments, in which the allocation of funds for each selected asset is very similar. This may result in a focus on the low-risk and low-yield solutions. To solve this problem, an improved multi-dimensional mapping coding scheme is proposed in this paper. This new coding scheme is integrated into the decomposition based multi-objective evolutionary algorithm (MOEA/D). The algorithm is then applied to some test data, with the asset size ranging from 31 to 255, and the experimental results have indicated that the improved MDM coding scheme can significantly improve the performance comparing to the basic MDM coding scheme.

The performance of traditional multiobjective evolutionary algorithms (MOEAs) often deteriorates rapidly as the number of decision variables increases. While some efforts were made to design new algorithms by adapting existing techniques to large-scale single-objective optimization to the MOEA context, the specific difficulties that may arise from large-scale multiobjective optimization have rarely been studied. In this paper, the exclusive challenges along with the increase of the number of variables of a multiobjective optimization problem (MOP) are examined empirically, and the popular benchmarks are categorized into three groups accordingly. Problems in the first category only require MOEAs to have stronger convergence, and can thus be mitigated using techniques employed in large-scale single-objective optimization. Problems that require MOEAs to have stronger diversification but ignore a correlation between position and distance functions are grouped as the second. The rest of the problems that pose a great challenge to the balance between diversification and convergence by considering a correlation between position and distance functions are grouped as the third. While existing large-scale MOEAs perform well on the problems in the first two categories, they suffer a significant loss when applied to those in the third category. To solve large-scale MOPs in this category, we have developed a novel indicator-based algorithm with an enhanced diversification mechanism. The proposed algorithm incorporates a new solution generator with an external archive, thus forcing the search toward different subregions of the Pareto front using a dual local search mechanism. The results obtained by applying the proposed algorithm to a wide variety of problems (108 instances in total) with up to 8192 variables demonstrate that it outperforms eight state-of-the-art approaches on the examined problems in the third category and show its advantage in the balance between diversification and convergence.

Combining machine learning techniques has shown great potentials in evolutionary optimization since the domain knowledge of an optimization problem, if well learned, can be a great help for creating high-quality solutions. However, existing learning-based multiobjective evolutionary algorithms (MOEAs) spend too much computational overhead on learning. To address this problem, we propose a learning-based MOEA where an online learning algorithm is embedded within the evolutionary search procedure. The online learning algorithm takes the stream of sequentially generated solutions along the evolution as its training data. It is noted that the stream of solutions are temporal, dependent, nonstationary, and nonstatic. These data characteristics make existing online learning algorithm not suitable for the evolution data. We hence modify an existing online agglomerative clustering algorithm to accommodate these characteristics. The modified online clustering algorithm is applied to adaptively discover the structure of the Pareto optimal set; and the learned structure is used to guide new solution creation. Experimental results have shown significant improvement over four state-of-the-art MOEAs on a variety of benchmark problems.

This paper provides an overview of developments on termination conditions in evolutionary algorithms (EAs). It seeks to give a representative picture of the termination conditions in EAs over the past decades, segment the contributions of termination conditions into progress indicators and termination criteria. With respect to progress indicators, we consider a variety of indicators, in particular in convergence indicators and diversity indicators. With respect to termination criteria, this paper reviews recent research on threshold strategy, statistical inference, i.e., Kalman filters, as well as Fuzzy methods, and other methods. Key developments on termination conditions over decades include: (i) methods of judging the algorithm's search behavior based on statistics, and (ii) methods of detecting the termination based on different distance formulations.

In this paper, we proposed a generative model for feature selection under the unsupervised learning context. The model assumes that data are independently and identically sampled from a finite mixture of Student's distributions, which can reduce the sensitiveness to outliers. Latent random variables that represent the features' salience are included in the model for the indication of the relevance of features. As a result, the model is expected to simultaneously realize clustering, feature selection, and outlier detection. Inference is carried out by a tree-structured variational Bayes algorithm. Full Bayesian treatment is adopted in the model to realize automatic model selection. Controlled experimental studies showed that the developed model is capable of modeling the data set with outliers accurately. Furthermore, experiment results showed that the developed algorithm compares favorably against existing unsupervised probability model-based Bayesian feature selection algorithms on artificial and real data sets. Moreover, the application of the developed algorithm on real leukemia gene expression data indicated that it is able to identify the discriminating genes successfully.

In multiobjective evolutionary algorithms, how to select the optimal solutions from the offspring candidate set significantly affects the optimization process. Usually, the selection process is largely based on the real objective values or surrogate model estimating objective values. However, these selection processes are very time consuming sometimes, especially for some real optimization problems. Recently, some researches began to employ supervised classification to assist offspring selection, but these works are difficult to prepare the exact positive and negative samples or time consuming of parameter tuning problems. In order to solve these disadvantages, we propose a decomposition based multiobjective evolutionary algorithm with semi-supervised classification. This approach using random sampling and non-dominated sorting to construct semi supervised classifier. In each generation, a set of candidate solutions are generated for each subproblem and only good solutions are reserved by classifier. If there is more than one good solutions, we calculate each of good solutions by real objective function and choose the best one as the offspring solution. Based on the typical decomposition based multiobjective evolutionary algorithm MOEA/D, we design algorithm framework through integrating the novel offspring selection process based on semi-supervised classification. Experiments show that the proposed algorithm performs best in most test cases and improves the performance of MOEA/D.

The multiobjective evolutionary algorithm based on decomposition (MOEA/D) converts a multiobjective optimization problem into a set of single-objective subproblems, and tackles them simultaneously. In MOEA/D, the offspring generation is a crucial part to increase the convergence of the algorithm and maintain the diversity of the solution set. Currently, the majority of reproduction operators consider the quality of neighborhood exploration, i.e., the capability to distribute along the population structure, while few operators have good capability for subproblem exploitation, i.e., the ability to push solutions forward along the subproblems. To address this issue in this paper, we introduce one of the derivative-free optimization methods, Nelder-Mead simplex (NMS) method, to MOEA/D to accelerate the algorithm convergence. The NMS operator is combined with a differential evolution (DE) operator in the offspring generation. The comparison study demonstrates that calling the NMS operator occasionally can help to accelerate the convergence.

A regularity model-based multiobjective estimation of distribution algorithm (RM-MEDA) has been proposed for continuous multiobjective optimization problems. Generating promising solutions to approximate the population is significant to RM-MEDA. In the reproduction of RM-MEDA, it adopts a Latin square design strategy to sample points in the latent space that is extended to cover the whole Pareto set. However, the setting of the extension scale is problem-dependent to some extent. To circumvent this issue, we propose a differential evolution based sampling (DES) scheme for RM-MEDA. DES mutates the projections of the parent solutions in the latent space to generate promising candidate offspring solutions. The empirical experiment results have shown the significant advantages of the DES scheme comparing to the Latin square design.

The estimation of distribution algorithm is widely used to solve global optimization problems in recent years. The basic idea is using machine learning methods to extract relevant features of the search space among the selected individuals and to construct a probabilistic model for sampling new solutions. As we know, EDAs mainly focus on the global distribution information of population and are lack of solution location information. In this paper, we extend our previous work to propose a new EDA guided by the mean shift method, which is originally proposed as a density estimation method and is used as a local search method in this paper. In the new approach, at first a set of candidate solutions are generated by EDA. Then the mean shift method is used to refine some good parent solutions. Finally the sampled candidate solutions and the refined solutions are combined to form the offspring solutions. By this way, the global distribution information and the solution location information are used in offspring reproduction. We apply the new approach to a set of test instances and the experiment results indicate that the new algorithm can obtain good performance in most functions with a faster convergence rate.

Spectral clustering algorithms have been playing an important role in solving many problems in pattern recognition and image processing. As a well-known spectral clustering algorithm, Normalized Cut has been proved powerful in image segmentation and data clustering. Morever spectral clustering has shown to be more effective in finding clusters than many traditional algorithms such as k-means. However, how to decide the number of clusters is always a crucial problem we confront. It's just yet acknownledge that evolutionary algorithms have a powerful ability to solve such optimization problems. In this paper, we apply a Validity Measure for Fuzzy Clustering(VMFC) to determine the cluster number in spectral clustering with the Particle Swarm Optimization selecting the optimal number of clusters from several possible choices.

Multiobjective evolutionary algorithms based on decomposition (MOEA/D) decompose a multiobjective optimization problem into a set of simple optimization subproblems and solve them in a collaborative manner. A replacement scheme, which assigns a new solution to a subproblem, plays a key role in balancing diversity and convergence in MOEA/D. This paper proposes a global replacement scheme which assigns a new solution to its most suitable subproblems. We demonstrate that the replacement neighborhood size is critical for population diversity and convergence, and develop an approach for adjusting this size dynamically. A steady-state algorithm and a generational one with this approach have been designed and experimentally studied. The experimental results on a number of test problems have shown that the proposed algorithms have some advantages.

A decomposition approach decomposes a multiobjective optimization problem into a number of scalar objective optimization subproblems. It plays a key role in decomposition-based multiobjective evolutionary algorithms. However, many widely used decomposition approaches, originally proposed for mathematical programming algorithms, may not be very suitable for evolutionary algorithms. To help decomposition-based multiobjective evolutionary algorithms balance the population diversity and convergence in an appropriate manner, this letter proposes to impose some constraints on the subproblems. Experiments have been conducted to demonstrate that our proposed constrained decomposition approach works well on most test instances. We further propose a strategy for adaptively adjusting constraints by using information collected from the search. Experimental results show that it can significantly improve the algorithm performance.

Under mild conditions, the Pareto front (Pareto set) of a continuous m-objective optimization problem forms an (m - 1)-dimensional piecewise continuous manifold. Based on this property, this paper proposes a self-organizing multiobjective evolutionary algorithm. At each generation, a self-organizing mapping method with (m - 1) latent variables is applied to establish the neighborhood relationship among current solutions. A solution is only allowed to mate with its neighboring solutions to generate a new solution. To reduce the computational overhead, the self-organizing training step and the evolution step are conducted in an alternative manner. In other words, the self-organizing training is performed only one single step at each generation. The proposed algorithm has been applied to a number of test instances and compared with some state-of-the-art multiobjective evolutionary methods. The results have demonstrated its advantages over other approaches.

Decomposition-based multiobjective evolutionary algorithms (MOEAs) decompose a multiobjective optimization problem into a set of scalar objective subproblems and solve them in a collaborative way. A naïve way to distribute computational effort is to treat all the subproblems equally and assign the same computational resource to each subproblem. This paper proposes a generalized resource allocation (GRA) strategy for decomposition-based MOEAs by using a probability of improvement vector. Each subproblem is chosen to invest according to this vector. An offline measurement and an online measurement of the subproblem hardness are used to maintain and update this vector. Utility functions are proposed and studied for implementing a reasonable and stable online resource allocation strategy. Extensive experimental studies on the proposed GRA strategy have been conducted.

In multiobjective evolutionary algorithms, most selection operators are based on the objective values or the approximated objective values. It is arguable that the selection in evolutionary algorithms is a classification problem in nature, i.e., selection equals to classifying the selected solutions into one class and the unselected ones into another class. Following this idea, we propose a classification based preselection for multiobjective evolutionary algorithms. This approach maintains two external populations: one is a positive data set which contains a set of `good' solutions, and the other is a negative data set contains a set of `bad' solutions. In each generation, the two external populations are used to train a classifier firstly, then the classifier is applied to filter the newly generated candidate solutions and only the ones labeled as positive are kept as the offspring solutions. The proposed preselection is integrated into the Pareto domination based algorithm framework in this paper. A systematic empirical study on the influence of different classifiers and different reproduction operators has been done. The experimental results indicate that the classification based preselection can improve the performance of Pareto domination based multiobjective evolutionary algorithms.

The decomposition based multiobjective evolutionary algorithm, denoted as MOEA/D, is an open framework for multiobjective optimization. This paper addresses the reproduction operation in MOEA/D. Generally, the solutions from a neighborhood of a subproblem are chosen as the mating pool for offspring reproduction. Since the Pareto set of an MOP shows some kind of structure in the decision space, the newly generated solutions based on the mating pool are arguable more likely to distribute along the population structure, which is called neighborhood exploration, and less likely to push a solution forward along the subproblem, which is called subproblem exploitation. To balance neighborhood exploration and subproblem exploitation, we propose to utilize both history and neighbor solutions for offspring reproduction. This idea is implemented through two operators based on the multivariate Gaussian distribution model, one is based on neighbor solutions and the other is based on previously visited solutions. When generating a new trial solution for a subproblem, one of the two operators is chosen with a probability. The proposed reproduction strategy is embedded in the MOEA/D framework and applied to a test suite. The comparison study has demonstrated that the new reproduction strategy is promising.

Image segmentation plays a key role in many fields such as image processing and recognition. Although various segmentation methods have been proposed in recent decades, most of these methods are based on only a single feature space. How to combine various features to image segmentation is a challenge problem. To address this problem, the authors propose to combine different features based on evolutionary multiobjective optimisation. Two optimisation objectives, which are based on colour and texture features, respectively, are therefore designed for image segmentation. The experiments show that the author's method is able to combine multiple features for image segmentation successfully.

In this letter, we propose an antinoise method for hyperspectral unmixing. In the antinoise method, all noises are addressed. The following techniques are applied: 1) an endmember dictionary is constructed first to initialize the solution; 2) an approximated L0norm constraint is employed to prune the dictionary and fulfill the sparse coding; and 3) the Itakura-Saito divergence, instead of the Square of Euclidean Distance divergence, is utilized to construct a novel optimization function. The experimental results on both synthetic and real hyperspectral data sets demonstrate the efficacy of the proposed method.

It is well known that in evolutionary algorithms (EAs), different reproduction operators may be suitable for different problems or in different running stages. To improve the algorithm performance, the ensemble of multiple operators has become popular. Most ensemble techniques achieve this goal by choosing an operator according to a probability learned from the previous experience. In contrast to these ensemble techniques, in this paper we propose a cheap surrogate model-based multioperator search strategy for evolutionary optimization. In our approach, a set of candidate offspring solutions are generated by using the multiple offspring reproduction operators, and the best one according to the surrogate model is chosen as the offspring solution. Two major advantages of this approach are: 1) each operator can generate a solution for competition compared to the probability-based approaches and 2) the surrogate model building is relatively cheap compared to that in the surrogate-assisted EAs. The model is used to implement multioperator ensemble in two popular EAs, that is, differential evolution and particle swarm optimization. Thirty benchmark functions and the functions presented in the CEC 2013 are chosen as the test suite to evaluate our approach. Experimental results indicate that the new approach can improve the performance of single operator-based methods in the majority of the functions.

In an estimation of distribution algorithm (EDA), global population distribution is modeled by a probabilistic model, from which new trial solutions are sampled, whereas individual location information is not directly and fully exploited. In this paper, we suggest to combine an EDA with cheap and expensive local search (LS) methods for making use of both global statistical information and individual location information. In our approach, part of a new solution is sampled from a modified univariate histogram probabilistic model and the rest is generated by refining a parent solution through a cheap LS method that does not need any function evaluation. When the population has converged, an expensive LS method is applied to improve a promising solution found so far. Controlled experiments have been carried out to investigate the effects of the algorithm components and the control parameters, the scalability on the number of variables, and the running time. The proposed algorithm has been compared with two state-of-the-art algorithms on two test suites of 27 test instances. Experimental results have shown that, for simple test instances, our algorithm can produce better or similar solutions but with faster convergence speed than the compared methods and for some complicated test instances it can find better solutions.

In evolutionary algorithms, the reproduction operators play an important role. It is arguable that different operators may be suitable for different kinds of problems. Therefore, it is natural to combine multiple operators to achieve better performance. To demonstrate this idea, in this paper, we propose an MOEA/D with multiple differential evolution mutation operators called MOEA/D-MO. MOEA/D aims to decompose a multiobjective optimization problem (MOP) into a number of single objective optimization problems (SOPs) and optimize those SOPs simultaneously. In MOEA/D-MO, we combine multiple operators to do reproduction. Three mutation strategies with randomly selected parameters from a parameter pool are used to generate new trial solutions. The proposed algorithm is applied to a set of test instances with different complexities and characteristics. Experimental results show that the proposed combining method is promising.

The evolutionary algorithms are usually criticized for their slow convergence. To address this weakness, a variety of strategies have been proposed. Among them, the metamodel or surrogate based approaches are promising since they replace the original optimization objective by a metamodel. However, the metamodel building itself is expensive and therefore the metamodel based evolutionary algorithms are commonly applied to expensive optimization. In this paper, we propose an alternative metamodel, named locally weighted metamodel (LWM), for the pre-selection in evolutionary optimization. The basic idea is to estimate the objective values of candidate offspring solutions for an individual, and choose the most promising one as the offspring solution. Instead of building a global model as many other algorithms do, a LWM is built for each candidate offspring solution in our approach. The LWM based pre-selection is implemented in a multi-operator based evolutionary algorithm, and applied to a set of test instances with different characteristics. Experimental results show that the proposed approach is promising.

This paper studies the replacement schemes in MOEA/D and proposes a new replacement named global replacement. It can improve the performance of MOEA/D. Moreover, trade-offs between convergence and diversity can be easily controlled in this replacement strategy. It also shows that different problems need different trade-offs between convergence and diversity. We test the MOEA/D with this global replacement on three sets of benchmark problems to demonstrate its effectiveness.

In this paper, we propose a clustering based multiobjective evolutionary algorithm (CLUMOEA) to deal with the multiobjective optimization problems with irregular Pareto front shapes. CLUMOEA uses a k-means clustering method to discover the population structure by partitioning the solutions into several clusters, and it only allows the solutions in the same cluster to do the reproduction. To reduce the computational cost and balance the exploration and exploitation, the clustering process and evolutionary process are integrated together and they are performed simultaneously. In addition to the clustering, CLUMOEA also uses a distance tournament selection to choose the more similar mating solutions to accelerate the convergence. Besides, a cosine nondominated selection method considering the location and distance information of the solutions are further presented to construct the final population with good diversity. The experimental results show that, compared with some state-of-the-art algorithms, CLUMOEA has significant advantages on dealing with the given test problems with irregular Pareto front shapes.

In this paper, we proposed a generative graphical model for unsupervised robust feature selection. The model assumes that the data are independent and identically sampled from a finite mixture of Student-t distribution for dealing with outliers. The Student t-distribution works as the building block for robust clustering and outlier detection. Random variables that represent the features' saliency are included in the model for feature selection. As a result, the model is expected to simultaneously realise unsupervised clustering, feature selection and outlier detection. The inference is carried out by a tree-structured variational Bayes (VB) algorithm. The feature selection capability is realised by estimating the feature saliencies associated with the features. The adoption of full Bayesian treatment in the model realises automatic model selection. Experimental studies showed that the developed algorithm compares favourably against existing unsupervised Bayesian feature selection algorithm in terms of commonly-used internal and external cluster validity indices on controlled experimental settings and benchmark data sets. The controlled experimental study also showed that the developed algorithm is capable of exposing the outliers and finding the optimal number of components (model selection) accurately.

Undersegmentation or oversegmentation is a challenge faced in image segmentation methods, and it is extreme important to determine the optimal number of regions (clusters) of an image in real‐world applications. In this study, we introduce an adaptive strategy to do so. The basic idea is to firstly oversegment an image by using the Mean‐shift (MS) method, and then segment the obtained oversegmented results by using an evolutionary algorithm. In the second stage, a feature is extracted for each region obtained by the MS method, and a new fitness function is designed to determine the optimal number of clusters. The adaptive approach is applied to a variety of images, and the experimental results show that our method is both efficient and effective for image segmentation.

It is well known that the linear mixture model (LMM) is attracting much attention due to its simplicity. However, some theoretical analysis reveals that the traditional LMM also impedes the improvement of blind spectral unmixing. For this reason, we propose a novel blind spectral unmixing method (NBSUM) in this letter. NBSUM utilizes the conjugate gradient to calculate end-member spectral and abundance, which can not only overcome some shortcomings of the traditional LMM but also provide more accurate results. NBSUM is compared with some state-of-the-art approaches on both synthetic and real hyperspectral data sets, and the experimental results demonstrate the efficacy of the proposed method.

This paper investigates how to use prediction strategies to improve the performance of multiobjective evolutionary optimization algorithms in dealing with dynamic environments. Prediction-based methods have been applied to predict some isolated points in both dynamic single objective optimization and dynamic multiobjective optimization. We extend this idea to predict a whole population by considering the properties of continuous dynamic multiobjective optimization problems. In our approach, called population prediction strategy (PPS), a Pareto set is divided into two parts: a center point and a manifold. A sequence of center points is maintained to predict the next center, and the previous manifolds are used to estimate the next manifold. Thus, PPS could initialize a whole population by combining the predicted center and estimated manifold when a change is detected. We systematically compare PPS with a random initialization strategy and a hybrid initialization strategy on a variety of test instances with linear or nonlinear correlation between design variables. The statistical results show that PPS is promising for dealing with dynamic environments.

Differential evolution (DE) is a kind of evolutionary algorithms (EAs), which are population based heuristic global optimization methods. EAs, including DE, are usually criticized for their slow convergence comparing to traditional optimization methods. How to speed up the EA convergence while keeping its global search ability is still a challenge in the EA community. In this paper, we propose a differential evolution method with an orthogonal local search (OLSDE), which combines orthogonal design (OD) and EA for global optimization. In each generation of OLSDE, a general DE process is used firstly, and then an OD based local search is utilized to improve the quality of some solutions. The proposed OLSDE is applied to a variety of test instances and compared with a basic DE method and an orthogonal based DE method. The experimental results show that OLSDE is promising for dealing with the given continuous test instances.

How to set the cluster number plays a key role in many clustering applications. To address this issue, this study introduces an automatic clustering method based on evolutionary algorithms (EAs). The basic idea is to convert a clustering problem into a global optimisation problem and tackle it by an EA. A new validity index, which balances the inter‐cluster consistency and the intra‐cluster consistency, is proposed to be the objective function. Three adaptive coding schemes, which can deal with variable‐length optimisation problems by using a fixed‐length chromosome, are designed to detect the cluster number automatically. The validity index and adaptive coding schemes are incorporated in an EA for automatic clustering. The authors approach is compared with some widely used validity indices and an adaptive coding scheme on some artificial data sets and two real‐world problems. The experimental results suggest that their method not only successfully detects the correct cluster numbers but also achieve stable results for most of test problems.

Many real world applications require optimizing multiple objectives simultaneously. Multiobjective evolutionary algorithm based on decomposition (MOEA/D) is a new framework for dealing with such kind of multiobjective optimization problems (MOPs). MOEA/D focuses on how to maintain a set of scalarized sub-problems to approximate the optimum of a MOP. This paper addresses the offspring reproduction operator in MOEA/D. It is arguable that, to design efficient offspring generators, the properties of both the algorithm to use and the problem to tackle should be considered. To illustrate this idea, a generator based on multivariate Gaussian models is proposed under the MOEA/D framework in this paper. In the new generator, both the local and global population distribution information is extracted by a set of Gaussian distribution models; new trial solutions are sampled from the probability models. The proposed approach is applied to a set of benchmark problems with complicated Pareto sets. The comparison study shows that the offspring generator is promising for dealing with continuous MOPs.

Network connectivity, as a fundamental issue in a wireless sensor network(WSN), has been receiving considerable attention during the past decade. Most works focused on how to maintain full connectivity while conserving network resources. However, full connectivity is actually a sufficient but not necessary condition for many WSNs to communicate and function successfully. In addition, full connectivity requires high-demand in network cost as more sensors will be needed. Further, it is subject to high energy consumption and communication interference as higher communication power might be needed to connect the most isolated sensors. In view of this, this work investigates the hop distance in a randomly deployed WSN with partial network connectivity through modeling, analysis, and simulation perspectives. The results help in selecting critical network parameters for practical WSN designs of diverse WSN applications.

Dynamic constrained multi-objective differential evolution (DCMODE) is designed for solving constrained optimization problem (COP). Main feature presented in this paper is to construct dynamic multi-objective optimization problem (DMOP) from COP. The two evolved objectives are original function objective and violation objective. Constraints are controlled by dynamic environments, where the relaxed constraints boundaries are gradually tightened to original boundaries. After this dynamic process, DMOP solutions are close to COP solution. This new algorithm is tested on benchmark problems of special session at CEC2006 with 100% success rates of all problems. Compared with several state-of-the-art DE variants referred in this paper, our algorithm outperforms or performs similarly to them. The satisfactory results suggest that it is efficient and generic when handling inequality/equality constraints.

Constrained optimization problem (COP) is skillfully converted into dynamic constrained multi-objective optimization problem (DCMOP) in this paper. Then dynamic constrained multi-objective evolutionary algorithms (DCMOEAs) can be used to solve the COP problem by solving the DCMOP problem. Seemingly, a complex DCMOEA algorithm is used to solve a relatively simple COP problem. However, the DCMOEA algorithm can adopt Pareto domination to achieve a good trade off between fast converging and global searching, and therefore a DCMOEA algorithm can effectively solve a COP problem by solving the DCMOP problem. An instance of DCMOEA was used to to solve 13 widely used constraint benchmark problems, The experimental results suggest it outperforms or performs similarly to other state-of-the-art algorithms referred to in this paper. The efficient performance of the DCMOEA algorithm shows, to some extend, the DCMOP model works well.

Probabilistic models play a key role in an estimation of distribution algorithm(EDA). Generally, the form of a probabilistic model has to be chosen before executing an EDA. In each generation, the probabilistic model parameters will be estimated by training the model on a set of selected individuals and new individuals are then sampled from the probabilistic model. In this paper, we propose to use probabilistic models in a different way: firstly generate a set of candidate points, then find some as offspring solutions by a filter which is based on a nonparametric density estimation method. Based on this idea, we propose a nonparametric estimation of distribution algorithm (nEDA) for global optimization. The major differences between nEDA and traditional EDAs are (1) nEDA uses a generating filtering strategy to create new solutions while traditional EDAs use a model building-sampling strategy to generate solutions, and (2) nEDA utilizes a nonparametric density model with traditional EDAs usually utilize parametric density models. nEDA is compared with a traditional EDA which is based on Gaussian model on a set of benchmark problems. The preliminary experimental results show that nEDA is promising for dealing with global optimization problems.

Mobile agents are often used in wireless sensor networks for distributed target detection with the goal of minimizing the transmission of non-critical data that negatively affects the performance of the network. A challenge is to find optimal mobile agent routes for minimizing the data path loss and the sensors energy consumption as well as maximizing the data accuracy. Existing approaches deal with the objectives individually, or by optimizing one and constraining the others or by combining them into a single objective. This often results in missing “good” tradeoff solutions. Only few approaches have tackled the Mobile Agent-based Distributed Sensor Network Routing problem as a Multiobjective Optimization Problem (MOP) using conventional Multi-Objective Evolutionary Algorithms (MOEAs). It is well known that the incorporation of problem specific knowledge in MOEAs is a difficult task. In this paper, we propose a problem-specific MOEA based on Decomposition (MOEA/D) for optimizing the three objectives. Experimental studies have shown that the proposed problem-specific approach performs better than two conventional MOEAs in several WSN test instances.

Minimax optimization requires to minimize the maximum output in all possible scenarios. It is a very challenging problem to evolutionary computation. In this paper, we propose a surrogate-assisted evolutionary algorithm, Minimax SAEA, for tackling minimax optimization problems. In Minimax SAEA, a surrogate model based on Gaussian process is built to approximate the mapping between the decision variables and the objective value. In each generation, most of the new solutions are evaluated based on the surrogate model and only the best one is evaluated by the actual objective function. Minimax SAEA is tested on six benchmark problems and the experimental results show that Minimax SAEA can successfully solve five of them within 110 function evaluations.

Most existing multiobjective evolutionary algorithms aim at approximating the Pareto front (PF), which is the distribution of the Pareto-optimal solutions in the objective space. In many real-life applications, however, a good approximation to the Pareto set (PS), which is the distribution of the Pareto-optimal solutions in the decision space, is also required by a decision maker. This paper considers a class of multiobjective optimization problems (MOPs), in which the dimensionalities of the PS and the PF manifolds are different so that a good approximation to the PF might not approximate the PS very well. It proposes a probabilistic model-based multiobjective evolutionary algorithm, called MMEA, for approximating the PS and the PF simultaneously for an MOP in this class. In the modeling phase of MMEA, the population is clustered into a number of subpopulations based on their distribution in the objective space, the principal component analysis technique is used to estimate the dimensionality of the PS manifold in each subpopulation, and then a probabilistic model is built for modeling the distribution of the Pareto-optimal solutions in the decision space. Such a modeling procedure could promote the population diversity in both the decision and objective spaces. MMEA is compared with three other methods, KP1, Omni-Optimizer and RM-MEDA, on a set of test instances, five of which are proposed in this paper. The experimental results clearly suggest that, overall, MMEA performs significantly better than the three compared algorithms in approximating both the PS and the PF.

The Pareto front (Pareto set) of a continuous optimization problem with m objectives is a (m-1) dimensional piecewise continuous manifold in the objective space (the decision space) under some mild conditions. Based on this regularity property in the objective space, we have recently developed several multiobjective estimation of distribution algorithms (EDAs). However, this property has not been utilized in the decision space. Using the regularity property in both the objective and decision space, this paper proposes a simple EDA for multiobjective optimization. Since the location information has not efficiently used in EDAs, a combination of EDA and differential evolution (DE) is suggested for improving the algorithmic performance. The hybrid method and the pure EDA method proposed in this paper, and a DE based method are compared on several test instances. Experimental results have shown that the algorithm with the proposed strategy is very promising.

Under mild conditions, it can be induced from the Karush-Kuhn-Tucker condition that the Pareto set, in the decision space, of a continuous multiobjective optimization problem is a piecewise continuous (m - 1)-D manifold, where m is the number of objectives. Based on this regularity property, we propose a regularity model-based multiobjective estimation of distribution algorithm (RM-MEDA) for continuous multiobjective optimization problems with variable linkages. At each generation, the proposed algorithm models a promising area in the decision space by a probability distribution whose centroid is a (m - 1)-D piecewise continuous manifold. The local principal component analysis algorithm is used for building such a model. New trial solutions are sampled from the model thus built. A nondominated sorting-based selection is used for choosing solutions for the next generation. Systematic experiments have shown that, overall, RM-MEDA outperforms three other state-of-the-art algorithms, namely, GDE3, PCX-NSGA-II, and MIDEA, on a set of test instances with variable linkages. We have demonstrated that, compared with GDE3, RM-MEDA is not sensitive to algorithmic parameters, and has good scalability to the number of decision variables in the case of nonlinear variable linkages. A few shortcomings of RM-MEDA have also been identified and discussed in this paper.

In the above titled paper (ibid., vol. 12, no. 1, pp. 41-63, Feb. 08), Fig. 20 was wrong. Its replacement is presented here.

The Pareto optimal set of a continuous multiobjective optimization problem is a piecewise continuous manifold under some mild conditions.We have recently developed several multi-objective evolutionary algorithms based on this property. However, the modelling methods used in these algorithms are rather costly. In this paper, a cheap and effective modelling strategy is proposed for building the probabilistic models of promising solutions. A new criterion is proposed for measuring the convergence of the algorithm. The locality degree of each local model is adjusted according to the proposed convergence criterion. Experimental results show that the algorithm with the proposed strategy is very promising.

Multiobjective optimization problems with many local Pareto fronts is a big challenge to evolutionary algorithms. In this paper, two operators, biased initialization and biased crossover, are proposed to improve the global search ability of RM-MEDA, a recently proposed multiobjective estimation of distribution algorithm. Biased initialization inserts several globally Pareto optimal solutions into the initial population; biased crossover combines the location information of some best solutions found so far and globally statistical information extracted from current population. Experiments have been conducted to study the effects of these two operators.

In our previous work [1], it has been shown that the performance of multi-objective evolutionary algorithms can be greatly enhanced if the regularity in the distribution of Pareto-optimal solutions is used. This paper suggests a new hybrid multi-objective evolutionary algorithm by introducing a convergence based criterion to determine when the model-based method and when the genetics-based method should be used to generate offspring in each generation. The basic idea is that the genetics-based method, i. e., crossover and mutation, should be used when the population is far away from the Pareto front and no obvious regularity in population distribution can be observed. When the population moves towards the Pareto front, the distribution of the individuals will show increasing regularity and in this case, the model-based method should be used to generate offspring. The proposed hybrid method is verified on widely used test problems and our simulation results show that the method is effective in achieving Pareto-optimal solutions compared to two state-of-the-art evolutionary multi-objective algorithms: NSGA-II and SPEA2, and our pervious method in [1].

The Pareto optimal solutions to a multi-objective optimization problem often distribute very regularly in both the decision space and the objective space. Most existing evolutionary algorithms do not explicitly take advantage of such a regularity. This paper proposed a model-based evolutionary algorithm (M-MOEA) for bi-objective optimization problems. Inspired by the ideas from estimation of distribution algorithms, M-MOEA uses a probability model to capture the regularity of the distribution of the Pareto optimal solutions. The local principal component analysis (local PCA) and the least-squares method are employed for building the model. New solutions are sampled from the model thus built. At alternate generations, M-MOEA uses crossover and mutation to produce new solutions. The selection in M-MOEA is the same as in non-dominated sorting genetic algorithm-II (NSGA-II). Therefore, MOEA can be regarded as a combination of EDA and NSGA-II. The preliminary experimental results show that M-MOEA performs better than NSGA-II.

Dynamic optimisation problems are becoming increasingly important; meanwhile, progress in optimisation techniques and in computational resources are permitting the development of effective systems for dynamic optimisation, resulting in a need for objective methods to evaluate and compare different techniques. The search for effective techniques may be seen as a multi-objective problem, trading off time complexity against effectiveness; hence benchmarks must be able to compare techniques across the Pareto front, not merely at a single point. We propose benchmarks for the dynamic travelling salesman problem, adapted from the CHN-144 benchmark of 144 Chinese cities for the static travelling salesman problem. We provide an example of the use of the benchmark, and illustrate the information that can be gleaned from analysis of the algorithm performance on the benchmarks.

Many real world optimization problems are time-dependent and some of them can be modeled by the dynamic TSPs (DTSPs). A DTSP is harder than a general TSP, which is a NP-hard problem, because the city number and the cost matrix of a DTSP are time varying. Although DTSP is a very common and important model in real world systems, few literatures have discussed this related issues. There are many open questions about DTSP urgently needed to be answered. We first give a mathematical model and the optimization objective for DTSP. Then we discuss why evolutionary algorithms (EAs) are effective for solving DTSPs and give some key points for designing efficient DTSP EAs. By defining three dynamic operators, we proposed an evolutionary algorithm for DTSPs. The experiments show the new algorithm is suitable for solving DTSPs. At the end, we also give some preliminary ideas for reinforcing the efficiency of EAs for DTSPs.