Adversarial perturbations (APs) have become a great concern in image classification tasks. The most challenging branch, universal adversarial perturbations (UAPs), are exploited to fool most of the unseen samples. Such one-to-all perturbations have the merit of transferability, which has strong practical significance. In this paper, we firstly define the transferability gap and the algorithm stability of the UAP algorithm, and prove the relationship between them. In analyzing the UAP algorithm stability, we prove that the convergence domain of existing UAP algorithms with dynamic constraints is excessively small, which degrades the capacity of UAPs. Thus, we further propose a new expected constraint and prove that UAPs in the expected constraint suit any sample in a high probability. Besides, we propose a Stochastic Universal Adversarial Perturbation (SUAP) that involves additive noise and the expected constraint. Finally, by treating the proposed algorithm as a stochastic differential equation, we prove an upper bound of the UAP algorithm stability of SUAP, which decreases exponentially at the beginning and then increases with a sublinear rate to at most a fixed constant. Experimental results show that SUAP is aligned with our analysis.

Large Language Models (LLMs) have demonstrated remarkable generalization capabilities, but aligning their outputs with human preferences typically requires expensive supervised fine-tuning. Recent test-time methods leverage textual feedback to overcome this, but they often critique and revise a single candidate response, lacking a principled mechanism to systematically analyze, weigh, and synthesize the strengths of multiple promising candidates. Such a mechanism is crucial because different responses may excel in distinct aspects (e.g., clarity, factual accuracy, or tone), and combining their best elements may produce a far superior outcome. This paper proposes the Textual Self-Attention Network (TSAN), a new paradigm for test-time preference optimization that requires no parameter updates. TSAN emulates self-attention entirely in natural language to overcome this gap: it analyzes multiple candidates by formatting them into textual keys and values, weighs their relevance using an LLM-based attention module, and synthesizes their strengths into a new, preference-aligned response under the guidance of the learned textual attention. This entire process operates in a textual gradient space, enabling iterative and interpretable optimization. Empirical evaluations demonstrate that with just three test-time iterations on a base SFT model, TSAN outperforms supervised models like Llama-3.1-70B-Instruct and surpasses the current state-of-the-art test-time alignment method by effectively leveraging multiple candidate solutions.

The core challenge of high-dimensional and expensive black-box optimization (BBO) is how to obtain better performance faster with little function evaluation cost. The essence of the problem is how to design an efficient optimization strategy tailored to the target task. This paper designs a powerful optimization framework to automatically learn the optimization strategies from the target or cheap surrogate task without human intervention. However, current methods are weak for this due to poor representation of optimization strategy. To achieve this, 1) drawing on the mechanism of genetic algorithm, we propose a deep neural network framework called B2Opt, which has a stronger representation of optimization strategies based on survival of the fittest; 2) B2Opt can utilize the cheap surrogate functions of the target task to guide the design of the efficient optimization strategies. Compared to the state-of-the-art BBO baselines, B2Opt can achieve multiple orders of magnitude performance improvement with less function evaluation cost.

In real-world applications, spectral Graph Neural Networks (GNNs) are powerful tools for processing diverse types of graphs. However, a single GNN often struggles to handle different graph types—such as homogeneous and heterogeneous graphs—simultaneously. This challenge has led to the manual design of GNNs tailored to specific graph types, but these approaches are limited by the high cost of labor and the constraints of expert knowledge, which cannot keep up with the rapid growth of graph data. To overcome these challenges, we introduce AutoSGNN, an automated framework for discovering propagation mechanisms in spectral GNNs. AutoSGNN unifies the search space for spectral GNNs by integrating large language models with evolutionary strategies to automatically generate architectures that adapt to various graph types. Extensive experiments on nine widely-used datasets, encompassing both homophilic and heterophilic graphs, demonstrate that AutoSGNN outperforms state-of-the-art spectral GNNs and graph neural architecture search methods in both performance and efficiency.

With the urgent need to enhance the safety of LLM applications, there has been a growing focus on alignment training algorithms designed to keep large language models (LLMs) behaving in alignment with human values. Alignment training algorithms rely heavily on preference datasets, which are essential for finetuning LLMs to follow human preferences. However, generating and annotating these datasets is often costly and labor-intensive, making it critical to protect their copyright against unauthorized use. In this paper, we propose PreferCare, the first framework tailor-made for preference dataset copyright protection via watermark injection and verification. PreferCare comprises two consecutive stages: injection and verification. In the injection stage, a style transfer-based watermark signal and a bi-level watermark optimization process are designed to embed the watermark into the preference dataset. In the verification stage, we employ statistical tests to determine whether a suspect LLM has used the watermarked preference dataset without authorization. Extensive experiments on multiple popular LLMs have demonstrated that PreferCare achieves effectiveness, harmlessness, transferability, and robustness across diverse settings, and can successfully verify the watermark within 20 queries.

In the field of single objective optimization algorithms, learned evolutionary algorithms have achieved success in obtaining better performance than human-designed strategies. However, these learnable evolutionary algorithms are only applicable to single-objective optimization and cannot be applied to multi-objective optimization problems. In this study, we parameterize the mutation and crossover operators using the multi-head self-attention and the selection operator using a lightweight multilayer perceptron. We utilize the evolution strategy to train their parameters across multiple multi-objective optimization problems, resulting in the development of the Meta-Learned Multi-Objective Genetic Algorithm (Meta-MOGA). We compare Meta-MOGA with other multi-objective evolutionary algorithms on various test problems and evaluate its performance on untrained MOPs. The results demonstrate that our Meta-MOGA exhibits potential and generalizability.

Vision-Language Retrieval-Augmented Generation (VLRAG) systems have been widely applied to Large Vision-Language Models (LVLMs) to enhance their generation ability. However, the reliance on external multimodal knowledge databases renders VLRAG systems vulnerable to malicious poisoning attacks. In this paper, we introduce PoisonedEye, the first knowledge poisoning attack designed for VLRAG systems. Our attack successfully manipulates the response of the VLRAG system for the target query by injecting only one poison sample into the knowledge database. To construct the poison sample, we follow two key properties for the retrieval and generation process, and identify the solution by satisfying these properties. Besides, we also introduce a class query targeted poisoning attack, a more generalized strategy that extends the poisoning effect to an entire class of target queries. Extensive experiments on multiple query datasets, retrievers, and LVLMs demonstrate that our attack is highly effective in compromising VLRAG systems.

Zero-shot optimization aims to achieve both generalization and performance gains on solving previously unseen black-box optimization problems over SOTA methods without task-specific tuning. Pre-trained optimization models (POMs) address this challenge by learning a general mapping from task features to optimization strategies, enabling direct deployment on new tasks. In this paper, we identify three essential components that determine the effectiveness of POMs: (1) task feature modeling, which captures structural properties of optimization problems; (2) optimization strategy representation, which defines how new candidate solutions are generated; and (3) the feature-to-strategy mapping mechanism learned during pre-training. However, existing POMs often suffer from weak feature representations, rigid strategy modeling, and unstable training. To address these limitations, we propose EPOM, an enhanced framework for pre-trained optimization. EPOM enriches task representations using a cross-attention-based tokenizer, improves strategy diversity through deformable attention, and stabilizes training by replacing non-differentiable operations with a differentiable crossover mechanism. Together, these enhancements yield better generalization, faster convergence, and more reliable performance in zero-shot black-box optimization.

Foundation models for time series analysis (TSA) have attracted significant attention. However, challenges such as training data scarcity and imbalance continue to hinder their development. Inspired by complex dynamic system theories, we design a series-symbol data generation mechanism, enabling the unrestricted creation of high-quality time series data paired with corresponding symbolic expressions. To leverage series-symbol data pairs with strong correlations, we develop SymTime, a pre-trained foundation model for enhancing time series representation using symbolic information. SymTime demonstrates competitive performance across five major TSA tasks when fine-tunes with downstream tasks, rivaling foundation models pre-trained on real-world datasets. This approach underscores the potential of series-symbol data generation and pretraining mechanisms in overcoming data scarcity and enhancing task performance. The code is available at https://github.com/wwhenxuan/SymTime.

Deep learning models have become ubiquitous in myriad application areas due to their remarkable performance. This success would not be possible without the high-quality datasets for model training that are contributed by numerous data owners. Datasets have not only become valuable assets for data owners, but also contain sensitive information that raises concerns about privacy leakage. This gives rise to urgent needs for data owners to verify that model developers have stopped using their datasets immediately upon receiving data deletion requests, as mandated by the right to be forgotten regulation. In this paper, we provide an affirmative answer by proposingDuplexGuard: a novel framework for deletion right verification via a duplex watermarking approach. During watermark injection, for each owner's dataset,DuplexGuardgenerates duplex subsets of watermarked samples, i.e., the ambush subset and the surfacing subset. This duplex design is capable of offering a combination of watermark behaviors before and after data deletion, therefore allowing it to signify all potential dataset usage statuses.DuplexGuardalso proposes a new two-way handshake protocol for issuing data deletion requests to provide more robust and decisive verification for the deletion right. Extensive experiments on multiple benchmark datasets demonstrate thatDuplexGuardis effective and reliable in verification.

In real-world scenarios, most complex systems can be generally modelled as homogenous or heterogenous networks. Therefore, downstream tasks (e.g., node/graph classification, node clustering) based on these two types of graphs become ubiquitous and have drawn considerable attentions in recent years. Existing literatures on node classification mainly focuses on either homogeneous or heterogeneous graphs, while research on effectively carrying out node classification tasks on both types of graphs simultaneously still under-exploited. To fill this gap, we propose a universal Graph Neural Network architecture based on Subgraph and Subhypergraph (SS-GNN) with feature-enhanced strategy for node embedding on both homogeneous and heterogeneous graphs. Through construction of subgraph and subhypergraph with same-class nodes, our model can simultaneously deal with homogeneous and heterogeneous graphs. Graph attention modules are especially designed to embed subgraphs of same-class nodes to learn the internal topological structure and local community structure within the original graph. Additionally, to capture high-order features of graph and enhance the embedding representations of nodes, we also utilize hypergraph attention modules to embed subhypergraphs of same-class nodes. Unlike other approaches that rely on pre-defined meta-paths, our model can be readily applied to most real-world applications without requiring any domain knowledge. Finally, we conduct extensive experiments on three homogeneous and three heterogeneous real-world graphs to demonstrate the effectiveness of SS-GNN. The experimental results for node classification and clustering tasks not only show the superior performance of our proposed model compared to state-of-the-art, but also demonstrate its potentially good interpretability for graph analysis. This work may provide some enlightening insights to the study on universality of graph foundation model.

Pre-trained models are extensively embraced in deep learning, facilitating efficient fine-tuning for downstream user-specific tasks and yielding substantial computational savings. However, backdoor attacks present a significant security threat to downstream models constructed on corrupted pre-trained models, necessitating the implementation of effective countermeasures to mitigate this threat prior to deploying the models in safety-critical applications. This paper introducesPurifierand its advanced versionPurifier$^{+}$, the former of which mitigates backdoors in pre-trained models by aligning anomaly activation to normal activation, and the latter builds on this by making importance rating about activation patterns, boosting important activation patterns and suppressing unimportant activation patterns.PurifierandPurifier$^{+}$draw inspiration from the observation that anomaly activation patterns for backdoor triggers manifest across various perspectives such as channel-wise, cube-wise, and feature-wise, each exhibiting distinct levels of granularity. Crucially, the choice of alignment granularity plays a pivotal role in ensuring robustness and accuracy. In addressing this challenge,PurifierandPurifier$^{+}$demonstrate the ability to effectively thwart various categories of backdoor triggers devoid of requiring prior information about the specific backdoor attacks. Additionally, it offers a convenient and flexible deployment feature, namely, plug-and-play capability. The comprehensive experimental results demonstrate thatPurifierandPurifier$^{+}$outperform current methodologies regarding defense efficacy and accuracy in model inference with uncontaminated samples when subjected to a series of State-of-the-Art mainstream attacks.

Modeling continuous-time dynamics constitutes a foundational challenge, and uncovering inter-component correlations within complex systems holds promise for enhancing the efficacy of dynamic modeling. The prevailing approach of integrating graph neural networks with ordinary differential equations has demonstrated promising performance. However, they disregard the crucial signed information potential on graphs, impeding their capacity to accurately capture real-world phenomena and leading to subpar outcomes. In response, we introduce a novel approach: a signed graph neural ordinary differential equation, adeptly addressing the limitations of miscapturing signed information. Our proposed solution boasts both flexibility and efficiency. To substantiate its effectiveness, we seamlessly integrate our devised strategies into three preeminent graph-based dynamic modeling frameworks: graph neural ordinary differential equations, graph neural controlled differential equations, and graph recurrent neural networks. Rigorous assessments encompass three intricate dynamic scenarios from physics and biology, as well as scrutiny across four authentic real-world traffic datasets. Remarkably outperforming the trio of baselines, empirical results underscore the substantial performance enhancements facilitated by our proposed approach. Our code can be found at https://github.com/beautyonce/SGODE.

Class-imbalanced node classification tasks are prevalent in real-world scenarios. Due to the uneven distribution of nodes across different classes, learning high-quality node representations remains a challenging endeavor. The engineering of loss functions has shown promising potential in addressing this issue. It involves the meticulous design of loss functions, utilizing information about the quantities of nodes in different categories and the network’s topology to learn unbiased node representations. However, the design of these loss functions heavily relies on human expert knowledge and exhibits limited adaptability to specific target tasks. In this paper, we introduce a high-performance, flexible, and generalizable automated loss function search framework to tackle this challenge. Across 15 combinations of graph neural networks and datasets, our framework achieves a significant improvement in performance compared to state-of-the-art methods. Additionally, we observe that homophily in graph-structured data significantly contributes to the transferability of the proposed framework.

Zero-shot optimization involves optimizing a target task that was not seen during training, aiming to provide the optimal solution without or with minimal adjustments to the optimizer. It is crucial to ensure reliable and robust performance in various applications. Current optimizers often struggle with zero-shot optimization and require intricate hyperparameter tuning to adapt to new tasks. To address this, we propose a Pretrained Optimization Model (POM) that leverages knowledge gained from optimizing diverse tasks, offering efficient solutions to zero-shot optimization through direct application or fine-tuning with few-shot samples. Evaluation on the BBOB benchmark and two robot control tasks demonstrates that POM outperforms state-of-the-art black-box optimization methods, especially for high-dimensional tasks. Fine-tuning POM with a small number of samples and budget yields significant performance improvements. Moreover, POM demonstrates robust generalization across diverse task distributions, dimensions, population sizes, and optimization horizons. For code implementation, see https://github.com/ninja-wm/POM/.

In the realm of daily services, the deployment of deep neural networks underscores the paramount importance of their reliability. However, the vulnerability of these networks to adversarial attacks, primarily evasion-based, poses a concerning threat to their functionality. Common methods for enhancing robustness involve heavy adversarial training or leveraging learned knowledge from clean data, both necessitating substantial computational resources. This inherent time-intensive nature severely limits the agility of large foundational models to swiftly counter adversarial perturbations. To address this challenge, this paper focuses on the \textbf{Ra}pid \textbf{P}lug-\textbf{i}n \textbf{D}efender (\textbf{RaPiD}) problem, aiming to rapidly counter adversarial perturbations without altering the deployed model. Drawing inspiration from the generalization and the universal computation ability of pre-trained transformer models, we propose a novel method termed \textbf{CeTaD} (\textbf{C}onsidering Pr\textbf{e}-trained \textbf{T}ransformers \textbf{a}s \textbf{D}efenders) for RaPiD, optimized for efficient computation. \textbf{CeTaD} strategically fine-tunes the normalization layer parameters within the defender using a limited set of clean and adversarial examples. Our evaluation centers on assessing \textbf{CeTaD}'s effectiveness, transferability, and the impact of different components in scenarios involving one-shot adversarial examples. The proposed method is capable of rapidly adapting to various attacks and different application scenarios without altering the target model and clean training data. We also explore the influence of varying training data conditions on \textbf{CeTaD}'s performance. Notably, \textbf{CeTaD} exhibits adaptability across differentiable service models and proves the potential of continuous learning.

Fuzzy cognitive map (FCM) has been successfully applied to time series prediction due to its powerful dynamic system modeling and inference ability. Although many FCM-based methods have been proposed, their performance is far from satisfactory. The existing FCM-based methods have two limitations: First, the feature extraction of some methods is unreasonable and even leads to overfitting. Second, most methods ignore the local temporal features of time series. In this work, we propose a novel framework for time series prediction based on long short-term memory (LSTM) and high-order FCMs (HFCM) with an attention mechanism, termed LSTM-HFCM AM . To overcome the first limitation, different from other FCM-based methods that use Autoencoder to forcibly decompose each point of the time series into multiple points to represent the original sequences, we use a sliding window to preprocess the original time series and use the Encoder-Decoder framework to represent the sequences. This way makes the model has better generalization and interpretability. Then, HFCM is used to predict representations of sequences due to its powerful causal inference ability. Finally, self-attention is applied to restore the predicted sliding window data with focus, effectively improving the performance. To overcome the second limitation, we use the LSTM to learn the temporal features of time series fragments, thereby learning the local features of the whole time series. We validate the performance of LSTM-HFCM AM on twelve benchmark datasets. Compared with current methods, LSTM-HFCM AM has a maximum improvement of 51.02%. The experimental results demonstrate the effectiveness of LSTM-HFCM AM and overcome the above limitations.

Network structure evolves with time in the real world, and the discovery of changing communities in dynamic networks is an important research topic that poses challenging tasks. Most existing methods assume that no significant change occurs; namely, the difference between adjacent snapshots is slight. However, great change exists in the real world usually. The great change in the network will result in the community detection algorithms are difficulty obtaining valuable information from the previous snapshot, leading to negative transfer for the next time steps. This article focuses on dynamic community detection with substantial changes by integrating higher order knowledge from the previous snapshots to aid the subsequent snapshots. Moreover, to improve search efficiency, a higher order knowledge transfer strategy is designed to determine first-order and higher order knowledge by detecting the similarity of the adjacency matrix of snapshots. In this way, our proposal can keep the advantages of previous community detection results and transfer them to the next task. We conduct the experiments on four real-world networks, including the networks with great or minor changes. Experimental results in the low-similarity datasets demonstrate that higher order knowledge is more valuable than first-order knowledge when the network changes significantly and keeps the advantage even if handling the high-similarity datasets. Our proposal can also guide other dynamic optimization problems with great changes.

Fuzzy cognitive maps (FCMs) play a significant role in inferring causal relationships and modeling complex systems. In recent years, numerous FCM-based causal discovery methods have emerged for diverse causal inference scenarios. However, determining the causal relations that best represent the ground truth in a novel scenario remains a challenge, resulting in an abundant but noisy set of candidate causal relations. Addressing this challenge, we introduce a parameter-free model grounded in Bayesian and fuzzy theory to estimate the accurate causal relations of the real system from candidate causal relation datasets. We initially fuzzify the edge weights within candidate FCMs into distinct causal states. Assuming a probabilistic data model linking observed data (causal relations in candidate FCMs) to true causality, we employ Bayesian methods to transform model parameter solving into a maximum posterior probability estimation task. This approach yields an accurate probabilistic data model, facilitating precise estimation of genuine causality from datasets containing multiple candidate relations. This model aims to derive accurate causal relations from datasets populated with multiple candidate relations. Our method validated on 15 diverse causal datasets, constructed from real and synthetic gene regulatory data, exhibits superior accuracy in discerning causal relationships. These results also demonstrate the efficacy of our proposal: 1) stabilizing performance amidst noisy data; 2) resolving algorithmic diversity for precise causal inferences; 3) mitigating fluctuations from hyperparameters. The source code and dataset are available athttps://github.com/IngeTeng/Abundant-but-Noisy-FCMs.

Conducting functional connectivity analysis on functional magnetic resonance imaging (fMRI) data presents a significant and intricate challenge. Contemporary studies typically analyze fMRI data by constructing high-order functional connectivity networks (FCNs) due to their strong interpretability. However, these approaches often overlook temporal information, resulting in suboptimal accuracy. Temporal information plays a vital role in reflecting changes in blood oxygenation level-dependent signals. To address this shortcoming, we have devised a framework for extracting temporal dependencies from fMRI data and inferring high-order functional connectivity among regions of interest (ROIs). Our approach postulates that the current state can be determined by the FCN and the state at the previous time, effectively capturing temporal dependencies. Furthermore, we enhance FCN by incorporating high-order features through hypergraph-based manifold regularization. Our algorithm involves causal modeling of the dynamic brain system, and the obtained directed FC reveals differences in the flow of information under different patterns. We have validated the significance of integrating temporal information into FCN using four real-world fMRI datasets. On average, our framework achieves 12% higher accuracy than non-temporal hypergraph-based and low-order FCNs, all while maintaining a short processing time. Notably, our framework successfully identifies the most discriminative ROIs, aligning with previous research, and thereby facilitating cognitive and behavioral studies.

A fuzzy cognitive map (FCM) is a simple but effective tool for modeling and predicting time series. This article focuses on the problem of multivariate time series prediction (TSP), which is essential and challenging in data mining. Although several FCM-based approaches have been designed to solve this problem, their feature extraction module designed for single mode falls short in capturing the nonlinear spatiotemporal dependencies among variates, thereby resulting in low prediction accuracy in forecasting multivariate time series, which shows that the single mode learning is not enough. Therefore, in this article, we propose a joint spatiotemporal feature learning framework for multivariate TSP, where a mix-resolution spatial module consisting of multiple sparse autoencoders (SAEs) is designed to extract the feature series with different spatial resolutions, and a mix-order spatiotemporal module concluding multiple high-order FCMs (HFCMs) is designed to model the spatiotemporal dynamics of these feature series. Finally, the outputs of the two modules are concatenated to predict future values. We refer to this framework as the spatiotemporal FCM (STFCM). Especially, an efficient learning algorithm is designed to update the integral weights of STFCM based on the batch gradient descent algorithm when it deems necessary. We validate the performance of the STFCM on four real-world datasets. Compared with the existing state-of-the-art (SOTA) methods, the experimental results not only show the advantages of the two designed modules in the STFCM but also show the excellent performance of the STFCM.

Currently, the problem of uncovering complex network structure and dynamics from time series is prominent in many fields. Despite the recent progress in this area, reconstructing large-scale networks from limited data remains a tough problem. Existing works treat connections of nodes as continuous values, leaving a challenge of setting a proper cut-off value to distinguish whether the connections exist or not. Besides, their performances on large-scale networks are far from satisfactory. Considering the reconstruction error and sparsity as two objectives, this article proposes a subspace learning-based evolutionary multiobjective network reconstruction algorithm, called SLEMO-NR, to solve the aforementioned problems. In the evolutionary process, we assume that binary-coded individuals obey the Bernoulli distribution and can use the probability and natural parameter as alternative representations. Moreover, our approach utilizes the logistic principal component analysis (LPCA) to learn a subspace containing the features of the network structure. The offspring solutions are generated in the learned subspace and then can be mapped back to the original space via LPCA. Benefitting from the alternative representations, a preference-based local search operator (PLSO) is proposed to concentrate on finding solutions approximate to the true sparsity. The experimental results on synthetic networks and six real-world networks demonstrate that, due to the well-learned network structure subspace and the preference-based strategy, our approach is effective in reconstructing large-scale networks compared to six existing methods.

The problem of reconstructing nonlinear and complex dynamical systems from available data or time series is prominent in many fields, including engineering, physical, computer, biological, and social sciences. Many methods have been proposed to address this problem and their performance is satisfactory. However, none of them can reconstruct network structure from large-scale real-time streaming data, which leads to the failure of real-time and online analysis or control of complex systems. In this article, to overcome the limitations of current methods, we first extend the network reconstruction problem (NRP) to online settings, and then develop a follow-the-regularized-leader (FTRL)-Proximal style method to address the online complex NRP; we refer to it as Online-NR. The performance of Online-NR is validated on synthetic evolutionary game network reconstruction datasets and eight real-world networks. The experimental results demonstrate that Online-NR can effectively solve the problem of online network reconstruction with large-scale real-time streaming data. Moreover, Online-NR outperforms or matches nine state-of-the-art network reconstruction methods.

Due to the multilayer nature of real-world systems, the problem of inferring multilayer network structures from nonlinear and complex dynamical systems is prominent in many fields, including engineering, biological, physical, and computer sciences. Many network reconstruction methods have been proposed to address this problem, but none of them consider the similarities among network reconstruction tasks at different component layers, which are inspired by topology correlations and dynamic couplings among different component layers. This article develops an evolutionary multitasking multilayer network reconstruction framework to make use of the correlations among different component layers to improve the reconstruction performance; we refer to this framework as EM2MNR. In EM2MNR, the multilayer network reconstruction problem is first established as a multitasking multilayer network reconstruction problem, where the goal of each task is to reconstruct the network structure of a component layer. In addition, multitasking multilayer network reconstruction problems are high dimensional, but existing evolutionary multitasking algorithms may have poor performance when dealing with optimization problems with a high-dimensional search space. Inspired by the sparsity of multilayer networks, EM2MNR employs the restricted Boltzmann machine to extract low effective features from the original decision space and then decides whether to conduct knowledge transfer on these features. To verify the performance of EM2MNR, this article also designs a test suite for multilayer network reconstruction problems. The experimental results demonstrate the significant improvement obtained by the proposed EM2MNR framework on 96 multilayer network reconstruction problems.

Evolutionary multitask optimization (EMTO) has recently attracted widespread attention in the evolutionary computation community, which solves two or more tasks simultaneously to improve the convergence characteristics of tasks when individually optimized. Effective knowledge between tasks is transferred by taking advantage of the parallelism of population-based search. Without any prior knowledge about tasks, it is a challenging problem of how to adaptively transfer effective knowledge between tasks and reduce the impact of negative transfer in EMTO. However, these two issues are rarely studied simultaneously in the existing literature. Besides, in complex many-task environments, the potential relationships among individuals from highly diverse populations associated with tasks directly determine the effectiveness of cross-task knowledge transfer. Keeping those in mind, we propose a multitask evolutionary algorithm based on anomaly detection (MTEA-AD). Specifically, each task is assigned a population and an anomaly detection model. Each anomaly detection model is used to learn the relationship among individuals between the current task and the other tasks online. Individuals that may carry negative knowledge are identified as outliers, and candidate transferred individuals identified by the anomaly detection model are selected to assist the current task, which may carry common knowledge across the current task and other tasks. Furthermore, to realize the adaptive control of the degree of knowledge transfer, the successfully transferred individuals that survive to the next generation through the elitism are used to update the anomaly detection parameter. The fair competition between offspring and candidate transferred individuals can effectively reduce the risk of negative transfer. Finally, the empirical studies on a series of synthetic benchmarks and a practical study are conducted to verify the effectiveness of MTEA-AD. The experimental results demonstrate that our proposal can adaptively adjust the degree of knowledge transfer through the anomaly detection model to achieve highly competitive performance compared to several state-of-the-art EMTO methods.

The problem of inferring nonlinear and complex dynamical systems from available data is prominent in many fields, including engineering, biological, social, physical, and computer sciences. Many evolutionary algorithm (EA)-based network reconstruction methods have been proposed to address this problem, but they ignore several useful information of network structure, such as community structure, which widely exists in various complex networks. Inspired by the community structure, this article develops a community-based evolutionary multiobjective network reconstruction framework to promote the reconstruction performance of EA-based network reconstruction methods due to their good performance; we refer this framework as CEMO-NR. CEMO-NR is a generic framework and any population-based multiobjective metaheuristic algorithm can be employed as the base optimizer. CEMO-NR employs the community structure of networks to divide the original decision space into multiple small decision spaces, and then any multiobjective EA (MOEA) can be used to search for improved solutions in the reduced decision space. To verify the performance of CEMO-NR, this article also designs a test suite for complex network reconstruction problems. Three representative MOEAs are embedded into CEMO-NR and compared with their original versions, respectively. The experimental results have demonstrated the significant improvement benefiting from the proposed CEMO-NR in 30 multiobjective network reconstruction problems (MONRPs).

Fuzzy cognitive maps (FCMs) have been successfully applied to time series forecasting. However, it still remains challenging to handle multivariate long nonstationary time series, such as EEG data, which may change rapidly and have patterns of trend. To overcome this limitation, in this article, we propose a fast prediction model to deal with multivariate long nonstationary time series based on the combination of elastic net and high order fuzzy cognitive map (HFCM), which is termed as ElasticNetHFCM. The designed FCM models each variable by one node and the high-order FCM helps to capture the patterns of trend. A case study on predicting human actions through the Electroencephalogram (EEG) data in the form of multichannel long nonstationary time series is investigated based on the proposed prediction model. Specifically, we first predict EEG signals based on the historical data, then a 1D-convolutionary neural network (1D-CNN) is developed to classify the predicted time series. The experimental results on the Grasp-and-Lift dataset show that the proposal can predict the EEG data with lower prediction error compared with the other regression methods. The area under the curve scores obtained on the Grasp-and-Lift dataset by 1D-CNN are higher than those obtained by state-of-the-art classification methods for EEG data in most cases. These results illustrate that the proposal can predict and classify multivariate long nonstationary time series with high accuracy and efficiency.

The fuzzy cognitive map (FCM) is an effective tool for modeling and simulating complex dynamic systems. The research on the problem of learning FCM from the available time series is outstanding. Many batch FCM learning methods have been proposed to address this issue and the performance of these methods is satisfactory. However, these batch-learning methods are difficult to cope with large-scale data sets (for example, the memory in computers is not enough to store all instances) and real-time streaming data, leading to the failure of real-time and online analysis of complex systems. In this article, unlike the existing batch learning methods, such as evolutionary and regression-based methods, we first extend the FCM learning to an online setting, and then develop an effective algorithm based on a follow-the-regularized-leader (FTRL)-proximal style learning algorithm to address the online FCM learning problem, termed as OFCM. The performance of OFCM is validated on constructed benchmark data sets, including synthetic data sets, and gene regulatory network reconstruction data sets. The experimental results demonstrate the merits of OFCM, which can effectively solve online FCM learning problems.

Learning large-scale fuzzy cognitive maps (FCMs) with the sparse attribute automatically from time series without prior knowledge remains a challenging problem. Most existing automated learning methods were applied to learn small-scale FCMs, and the learned FCMs are much denser than the maps constructed by human experts. Learning FCMs is the procedure of judging whether there are connecting edges and determining the weights of connecting edges. Thus, we transform the problem of learning FCMs into a biobjective optimization problem with two objects of minimizing the measure error and the number of nonzero entries, respectively. To solve this optimization problem, a preference-based iterative thresholding evolutionary biobjective optimization algorithm for learning FCMs is proposed. The strategy focuses on the knee area of the Pareto front (PF) with preference on the solutions near the true sparsity. Moreover, an initialization operator based on random forest is proposed to increase the speed of convergence toward the PF. The experiments on large-scale synthetic data with varying sizes and densities and the application to the gene regulatory network reconstruction problem have been conducted to demonstrate that our proposal matches or exceeds the existing state-of-the-art FCM learning approaches in most cases in terms of four measures, namely, Data_Error, Out_of_Sample_Error, Model Error, and SS_Mean. The Data_Error obtained by the proposed method can achieve 3.07E-06 even when the number of nodes reaches 200. We also demonstrate the effectiveness of the proposed initialization operator and the preference-based strategy, which can result in a fast convergence speed and higher accuracy.

The problem of time series prediction based on fuzzy cognitive maps (FCMs) is unresolved. Although many methods have been proposed to cope with this issue, the performance of these methods is far from satisfactory. Traditional FCM-based predictors have three limitations. First, current feature extraction operators are incapable of learning good representations of original time series. Second, current methods use just the output of FCMs to predict the next value; they do not directly utilize the important information of the latent features. Third, current FCM-based predictors optimize each component individually, thereby leading to low prediction accuracy. For example, these methods first optimize the feature extraction operator and then learn the FCMs from the latent features; they do not simultaneously optimize the whole prediction model. In this article, we develop a framework based on a sparse autoencoder (SAE) and a high-order FCM (HFCM) to address the time series prediction problem; we refer this framework as SAE-FCM. To overcome the first limitation of current methods, an SAE is employed to extract features from original time series. Unlike current FCM-based predictors, our method combines the output of both the SAE and the HFCM to calculate the predicted value, thereby overcoming the second limitation of traditional FCM-based predictors. In an application of the idea of “fine tuning” in deep learning, the weights of SAE-FCM can be updated by the batch gradient descent method if the prediction errors are great. Thus, we can optimize SAE-FCM as a whole and overcome the third limitation. We validate the performance of SAE-FCM on ten datasets. Compared with the experimental results obtained by using state-of-the-art methods, the experimental results obtained by using SAE-FCM demonstrate the effectiveness of our method. Extensive experiments also show that SAE-FCM can effectively overcome the above limitations.

Classification-based optimization (CBO) is a recently proposed global optimization method which exhibits a good prospect. Previous studies have shown that CBO has a good performance when a function evaluation is cheap. However, its performance is far from being completely and precisely consummated in the optimization of an expensive black box function f. We term this task as multi-fidelity black-box optimization and develop MF-CBO, a novel method based on CBO. To the best of our knowledge, this is the first work to extend classification-based optimization methods to the multi-fidelity case. MF-CBO explores the space using the low fidelities and exploits the high fidelity on successively smaller regions. Experimental results demonstrate that MF-CBO outperforms the strategies which ignore the multi-fidelity information and other multi-fidelity methods on several real cases and synthetic examples.

Fuzzy cognitive maps (FCMs) are generally applied to model and analyze complex dynamical systems. However, the accuracy of population-based FCM learning algorithms is relatively low. Boosting is an effective method to improve the accuracy of any learning algorithm. To this end, we combine FCMs with boosting, termed as boosting fuzzy cognitive maps (BFCMs). The BFCM is an extension of FCMs and has a better performance on fast numerical reasoning than FCMs. In this paper, a real-coded genetic algorithm, which is a popular population-based learning algorithm, is improved on mutation operator and applied to learn the BFCM models, termed as RCGA-BFCM. In the experiments, RCGA-BFCM is applied to learn the BFCM from synthetic data with varying sizes and densities. The experimental results show that RCGA-BFCM can learn BFCMs with high accuracy from synthetic data. In addition, the performance of RCGA-BFCM is validated on the benchmark datasets DREAM3 and DREAM4. The experimental results show that RCGA-BFCM outperforms other learning algorithms obviously, which illustrates that RCGA-BFCM can reconstruct gene regulatory networks (GRNs) effectively.

Most of the existing community detection algorithms are based on vertex connectivity. While in many real networks, each vertex usually has one or more attributes describing its properties which are often homogeneous in a cluster. Such networks can be modeled as attributed graphs, whose attributes sometimes are equally important to topological structure in graph clustering. One important challenge is to detect communities considering both topological structure and vertex properties simultaneously. To this propose, a multiobjective evolutionary algorithm based on structural and attribute similarities (MOEA-SA) is first proposed to solve the attributed graph clustering problems in this paper. In MOEA-SA, a new objective named as attribute similarity is proposed and another objective employed is the modularity . A hybrid representation is used and a neighborhood correction strategy is designed to repair the wrongly assigned genes through making balance between structural and attribute information. Moreover, an effective multi-individual-based mutation operator is designed to guide the evolution toward the good direction. The performance of MOEA-SA is validated on several real Facebook attributed graphs and several ego-networks with multiattribute. Two measurements, namely density and entropy , are used to evaluate the quality of communities obtained. Experimental results demonstrate the effectiveness of MOEA-SA and the systematic comparisons with existing methods show that MOEA-SA can get better values of and in each graph and find more relevant communities with practical meanings. Knee points corresponding to the best compromise solutions are calculated to guide decision makers to make convenient choices.

Learning large-scale sparse fuzzy cognitive maps (FCMs) from observed data automatically without any prior knowledge remains an outstanding problem. Most existing methods are slow and have difficulty in dealing with large-scale FCMs, because of the large searching space. We develop a framework based on compressed sensing (CS), a convex optimization method, to learn large-scale sparse FCMs, called CS-FCM. Combining with the sparsity of FCMs, the task of learning FCMs is first decomposed into sparse signal reconstruction problems. The ability of CS to exactly recover the sparse signals provides CS-FCM the probability to exactly learn FCMs. In the experiments, CS-FCM is applied to learn both synthetic data with varying sizes and densities and real-life data. The results show that CS-FCM obtains good performance by just learning from a small amount of data. CS-FCM can effectively learn sparse FCMs with 1000 nodes and even more, which have one million weights to be determined. CS-FCM is also applied to reconstruct gene regulatory networks (GRNs), and the well-known benchmark datasets DREAM3 and DREAM4 are tested. The results show that CS-FCM also obtains high accuracy in reconstructing GRNs. CS-FCM establishes a paradigm for learning large-scale sparse FCMs with high accuracy.