Since the proposal of the differential evolution (DE) algorithm, most improvement efforts have concentrated on binomial crossover. However, we find that DE variants using exponential crossover can also outperform those using binomial crossover in optimization performance, provided that an appropriate parameter control scheme is employed. To develop a high-performance DE algorithm with exponential crossover, this article introduces an adaptive parameter control and selection strategy for DE with exponential crossover (APEX-DE). The main contributions of APEX-DE are summarized as follows: First, a novel adaptive parameter control (APC) technique is proposed, incorporating an automatically generated crossover rate $CR$ , a dual-stage scale factor $F$ generation mechanism, and a new adaptive strategy for the scale parameter $\sigma _{F}$ . Second, a novel selection mechanism is introduced to replace the classical DE selection, thereby enhancing the ability to escape from local optima. Third, a redirection strategy is developed to regenerate individuals and adjust their scale factor $F$ , enhancing evolutionary potential. APEX-DE is evaluated on a large test suite comprising 88 benchmark functions, as well as on a challenging uncrewed aerial vehicle (UAV) path-planning task in multithreat environments. Experimental evaluation confirms that APEX-DE delivers better performance than a broad set of state-of-the-art algorithms.

Takagi–Sugeno–Kang (TSK) fuzzy systems are valued for their interpretability and flexibility, yet scaling them to high-dimensional data remains challenging. On the antecedent side, computing firing strengths with the product T-norm often results in ineffective activations, undermining rule utilization. On the consequent side, classical least-squares and gradient-based training may generalize poorly without additional regularization. Thus, an efficient and effective training algorithm is required to strike a balance between generalization and precision. To overcome these challenges, we propose a high-dimensional adaptive uniform membership function (HDAUMF) that adapts the width parameter using a theoretically motivated lower bound on the Gaussian width. By incorporating an activation radius to regulate the activation distribution, HDAUMF improves the effectiveness of rule activation. Building on this, a variant of the Kaczmarz method is extended to learn consequent weights of multioutput TSK models, which is further enhanced by a dual residual strategy to accelerate convergence. To mitigate the computational burden of matrix operations, a partitioned update scheme is designed, giving rise to the dual residual iterated projection (DRIP) algorithm. Theoretical analysis shows that convergence under the dual residual strategy yields the minimum-norm least-squares solution, thereby supporting robust generalization. Combining these advancements, a novel iterated projection TSK fuzzy system (IPTSK) is developed. Numerical experiments validate that the proposed methods achieve competitive performance in tackling high-dimensional problems.

Radial basis function neural networks (RBFNNs) are widely applied due to their rapid modeling capabilities and efficient learning performance. However, when dealing with high-dimensional data, RBFNNs encounter two critical limitations: the hidden layer responses using Gaussian kernels suffer from ineffective activation and numeric underflow; and the estimation of output layer weights typically involves tedious parameter tuning and inefficient loading of high-dimensional feature matrices. To overcome these challenges, we first propose a dimensionality-adaptive Gaussian kernel function (DAGKF) equipped with a novel width adjustment mechanism that flexibly mitigates the numerical difficulties inherent in high-dimensional spaces. Moreover, to avoid processing entire feature matrices simultaneously, we introduce a multioutput coordinate descent (MOCD) algorithm that enables parallel computation across multioutput systems. Building upon MOCD, we further develop the joint residual MOCD (JRMOCD) algorithm, which incorporates a joint residual criterion for more effective weight estimation. The convergence of the JRMOCD algorithm is rigorously proven. Extensive experiments demonstrate the superior performance of the proposed methods, particularly in high-dimensional settings.

In the field of message-passing interface (MPI) program path coverage test case generation, evolutionary algorithms (EAs) have been frequently utilized to generate test cases. However, relying solely on EAs will incur excessive computational costs. In this article, we improve the efficiency and quality of MPI program path coverage test cases generated by EAs based on elite individual selection. First, data within the data domain is sampled and fitness is calculated to form a shared set. Then, the population data is initialized using EAs, and the fitness of individuals is predicted using the neighbor value sharing algorithm (NVSA). Subsequently, individuals are ranked using rank-based elite selection (RES). Finally, elite individuals are chosen through ranking to run the program and verify the generation of test cases. In order to reduce computational costs, data dimensionality reduction operations are added to the above process. We demonstrate that the proposed method can effectively generate test data and reduce test costs by comparing it with several excellent methods on seven representative MPI programs. Among them, NVSA has a maximum improvement of 42.2%, RES has a maximum improvement of 31.5%, dimensionality reduction can increase by 20.2%, and the overall method has a maximum improvement of 47.4%.

Analyzing irregularly sampled multivariate time series (ISMTS) data poses significant challenges, with such irregularities frequently occurring in contexts like the Industrial Internet of Things (IIoT). However, most existing methods are designed for regularly sampled data, limiting their effectiveness in handling such complexities. These traditional approaches struggle with misalignments across time and variate dimensions, often requiring extensive preprocessing that can result in information loss and the introduction of noise. Furthermore, they may incorrectly utilize processing units, such as variate or temporal tokens, leading to suboptimal performance. To tackle these challenges, we present the Adaptive Tokenization Transformer (ATFormer), an innovative model designed to improve the analysis of ISMTS data. ATFormer employs an adaptive mechanism to select appropriate tokens (temporal or variate) based on the unique characteristics of the time series data. By capturing each observation at a finer granularity, the model enhances token representation. A masked attention mechanism aggregates observations, creating more comprehensive tokens and embedding information consistently, thereby mitigating incomplete embeddings and noise. Additionally, ATFormer facilitates the formation of fine-grained tokens and performs coarse-grained self-attention operations, enhancing the model’s utilization of tokens through the interaction of information at different granularities. This multilevel processing allows the model to effectively capture detailed information while integrating broader features, ultimately improving overall performance. Our evaluations on two healthcare datasets and one human activity dataset demonstrate that ATFormer outperforms existing methods in analyzing ISMTS.

The majority of the unsupervised feature selection methods usually explore the first-order similarity of the data while ignoring the high-order similarity of the instances, which makes it easy to construct a suboptimal similarity graph. Furthermore, such methods, often are not suitable for performing feature selection due to their high complexity, especially when the dimensionality of the data is high. To address the above issues, a novel method, termed as unsupervised feature selection for high-order embedding learning and sparse learning (UFSHS), is proposed to select useful features. More concretely, UFSHS first takes advantage of the high-order similarity of the original input to construct an optimal similarity graph that accurately reveals the essential geometric structure of high-dimensional data. Furthermore, it constructs a unified framework, integrating high-order embedding learning and sparse learning, to learn an appropriate projection matrix with row sparsity, which helps to select an optimal subset of features. Moreover, we design a novel alternative optimization method that provides different optimization strategies according to the relationship between the number of instances and the dimensionality, respectively, which significantly reduces the computational complexity of the model. Even more amazingly, the proposed optimization strategy is shown to be applicable to ridge regression, broad learning systems and fuzzy systems. Extensive experiments are conducted on nine public datasets to illustrate the superiority and efficiency of our UFSHS.

Evolutionary sequential transfer optimization (ESTO), which aims to better optimize a target task using the knowledge extracted from a number of previously solved source tasks, has been gaining continually increasing research attention over the years. Particularly, solution-based ESTO (S-ESTO) that transfers task solutions has been receiving much popularity due to its ease of implementation and optimizer independency. However, the existing S-ESTO algorithms put much emphasis on utilizing source optimized solutions standing for global search experience without being aware of the potential of intermediate solutions that represent local optimization experience. Besides, most of them cannot take full advantage of the solution data from evolutionary search. In the light of the above, this study aims to develop a global and local search experience-based solution transfer technique to maximally release the potential of optimization experience hidden in the source tasks. First, a novel transferability metric named landscape encoding-based rank correlation (LERC) is developed. Then, we propose to divide the optimization experience into two classes: 1) global and 2) local search experience. Accordingly, by instantiating LERC into global and local versions, we develop two distinct transfer methods to exploit the global and local search experience, respectively. Finally, by combining the two transfer methods, we propose an S-ESTO algorithm that can transfer the global and local search experience simultaneously for maximum performance enhancement for the target task. Experiments conducted on a set of benchmark problems and a practical case study verify the efficacy of the proposed methods. The source code of our algorithm is available athttps://github.com/ccm831143/GL-LERC.

Fuzzy systems are widely used for their strong ability to handle uncertainty; however, they often struggle with high-dimensional problems. Here, a novel scalable model, the Multihead Takagi-Sugeno-Kang Fuzzy System (MHTSK), is proposed to efficiently handle high-dimensional data while maintaining strong interpretability. Unlike traditional fuzzy systems that construct a single antecedent using all features, MHTSK builds multiple sub-antecedents by randomly sampling subsets of features to address high-dimensional challenges. The rules generated by these sub-antecedents are jointly optimized, resulting in a naturally sparse consequent structure that enhances interpretability. Since the size and number of feature subsets directly influence the length and complexity of fuzzy rules, we propose an optimal parameter configuration scheme that balances: (1) preventing “numeric underflow” in high-dimensional data and (2) ensuring sufficient coverage of the original feature space. Furthermore, considering the randomness introduced by sampling, a rule extraction scheme is proposed to eliminate redundant rules. The refined rule base obtained after the extraction process is used to retrain the model, resulting in a new variant model called MHTSK_RE. The effectiveness of MHTSK and MHTSK_RE is demonstrated on eleven high-dimensional datasets. The results confirm their superior performance and outstanding interpretability in handling high-dimensional data.

Takagi-Sugeno-Kang (TSK) fuzzy system has been successfully applied to various practical problems due to its good performance and interpretability. One important prerequisite for the interpretability of a fuzzy system is a concise rule base. However, how to construct a concise TSK fuzzy system more transparently and adaptively from data remains a challenge. In this article, we propose a two-phase framework of constructing concise instance-based TSK fuzzy systems using a novel multi objective particle swarm optimization algorithm with a bimutation strategy (MOPSO-BM). In the first phase, a basic fuzzy system is constructed, where the rule antecedents are initialized based on some representative input instances selected by a sample filtering algorithm, and the consequent parameters are obtained by least squares estimation. Then in the second phase, the proposed MOPSO-BM algorithm is utilized to further reduce the number of fuzzy rules and fine-tune the consequent parameters, resulting in a set of fuzzy systems that achieve a good trade-off between accuracy and interpretability. Users have the flexibility to select the most suitable model based on their specific requirements, or simply via cross validation. Numerical experiments on twelve benchmark datasets have substantiated the superior performance of the proposed framework in designing high-precision and compact TSK fuzzy systems.

Fuzzy systems have been employed in many fields and achieved remarkable results. However, due to the inherent characteristics, there is still a challenge for them to deal with high-dimensional data which includes a large number of features and becomes very common in the current era of Big Data. Most of the literature employ the product, minimum, softmin and its adaptive versions to compute the firing strengths in fuzzy system modeling, where only the first two are standard T-norms conforming to the theoretical basis of fuzzy logic reasoning. But easy to cause numeric underflow and nondifferentiability are their disadvantages, respectively. To comply with fuzzy theory basis and alleviate the aforementioned predicament of dimensionality, this article investigates how to design high-dimensional Takagi–Sugeno–Kang (TSK) fuzzy system based on Dombi T-norm, which has not been explored for fuzzy system modeling to the best of our knowledge. We first build Dombi T-norm based TSK (DombiTSK) model, then an adaptive strategy is designed for the index parameter of Dombi T-norm to enhance the performance of DombiTSK, which results in so-called adaptive DombiTSK (ADMTSK) fuzzy system. To further improve the design and performance of ADMTSK, we give a novel membership function with positive lower bound to match the use of adaptive Dombi T-norm, which is constructed based on Gaussian membership function (GMF) and named as composite GMF (CGMF). The experiments are conducted on high-dimensional classification datasets with feature dimensions varying from 1024 to 120 432 to evaluate the proposed methodologies. Experimental results verify the advantages of adaptive Dombi T-norm and CGMF. The comparison results and statistical tests among ADMTSK and other state-of-the-art fuzzy algorithms demonstrate that our proposed model outperforms its rivals.

Utilizing generative adversarial networks (GANs) for oversampling imbalanced data has demonstrated its effectiveness. However, many GAN-based oversampling methods are confronted with a significant challenge, namely, mode collapse, especially when dealing with tabular imbalanced data. In this paper, two unique penalty terms are respectively incorporated into the loss functions of the discriminator and the generator of GAN to promote the generated samples to exhibit not just statistical but also spatial information consistency with the minority samples, thereby alleviating the issue of mode collapse. In contrast to other studies that fix the coefficient of the penalty terms, the optimal coefficients of the penalty terms are adaptively searched using a meta-learning approach, where Bayesian optimization is firstly employed to effectively handle situations involving small size of minority samples in the imbalanced data. We call the proposed model as META_GAN. Experimental results demonstrate that META_GAN outperforms alternative oversampling methods on general tabular and image imbalanced datasets and long-tailed datasets in terms of different metrics.

Kleinberg (2002) introduced three axioms to formalize the behavior of clustering algorithms and presented that no clustering algorithm satisfies them. However, this result usually is inconsistent with the practical experience of clustering algorithms. In this article, we reformulate these axioms to fill the gap and verify the existence of clustering algorithms satisfying the modified axioms when the point set is well-clusterable. In particular, the concept of $(\delta ,\varepsilon)$ -K segmentation is proposed to characterize the set that has the potential to be clustered well. Then, we verify the existence and the uniqueness of $(\delta ,\varepsilon)$ -K segmentation of a set, respectively. Next, we demonstrate that the $(\delta ,\varepsilon)$ -K segmentation is compatible with K-means, Min-Cut, DBSCAN, and several clustering internal evaluation (CIE) indexes. In addition, the ratio $\delta / \varepsilon $ can be treated as the measure of not only characterizing well-clusterable sets but also of evaluating the performance of clustering results, respectively.

Unsupervised feature selection (UFS) aims to learn an indicator matrix relying on some characteristics of the high-dimensional data to identify the features to be selected. However, traditional unsupervised methods perform only at the feature level, i.e., they directly select useful features by feature ranking. Such methods do not pay any attention to the interaction information with other tasks such as classification, which severely degrades their feature selection performance. In this article, we propose an UFS method which also takes into account the classification level, and selects features that perform well both in clustering and classification. To achieve this, we design a bi-level spectral feature selection (BLSFS) method, which combines classification level and feature level. More concretely, at the classification level, we first apply the spectral clustering to generate pseudolabels, and then train a linear classifier to obtain the optimal regression matrix. At the feature level, we select useful features via maintaining the intrinsic structure of data in the embedding space with the learned regression matrix from the classification level, which in turn guides classifier training. We utilize a balancing parameter to seamlessly bridge the classification and feature levels together to construct a unified framework. A series of experiments on 12 benchmark datasets are carried out to demonstrate the superiority of BLSFS in both clustering and classification performance.

Approximation ability is one of the most important topics in the field of neural networks (NNs). Feedforward NNs, activated by rectified linear units and some of their specific smoothed versions, provide universal approximators to convex as well as continuous functions. However, most of these networks are investigated empirically, or their characteristics are analyzed based on specific operation rules. Moreover, an adequate level of interpretability of the networks is missing as well. In this work, we propose a class of new network architecture, built with reusable neural modules (functional blocks), to supply differentiable and interpretable approximators for convex and continuous target functions. Specifically, first, we introduce a concrete model construction mechanism with particular blocks based on differentiable programming and the composition essence of the max operator, extending the scope of existing activation functions. Moreover, explicit block diagrams are provided for a clear understanding of the external architecture and the internal processing mechanism. Subsequently, the approximation behavior of the proposed network to convex functions and continuous functions is rigorously proved as well, by virtue of mathematical induction. Finally, plenty of numerical experiments are conducted on a wide variety of problems, which exhibit the effectiveness and the superiority of the proposed model over some existing ones.

Recurrent neural networks (RNNs) have been found to be a promising field of research for time series prediction, continuous-time system modeling, and discrete-time sequence data processing. The Elman network can be viewed as an RNN with a single layer of feedback connections and local storage units. However, the Elman network has two limitations that restrict its application. One is the overfitting phenomenon, which limits its generalization ability. The other is the complexity of the network structure, which makes its theoretical results incomplete and limits its practicality. The main contribution of this article is to propose a new Elman network model with a regularization term. First, we demonstrate the convergence of the backpropagation algorithm with weight decay regularization under some reasonable conditions. Second, contrary to the usual requirement for bounded weights, we show that such boundedness is no longer a necessary condition for convergence analysis. Third, we show that the requirements for learning rate and the stable set of error functions can also be relaxed. Finally, we provide simulation results to demonstrate the excellent performance of our proposed algorithm.

Dealing with the high-dimensional data is a challenging task for fuzzy systems. Some effective methods have been proposed to address this issue in recent years, but few of them have focused on both low-and high-dimensional data simultaneously. This study introduces an adaptive Yager T-norm and a composite exponential membership function (CEMF)-based Takagi–Sugeno–Kang (TSK) fuzzy model for solving both low-and high-dimensional problems, referred to as AYATSK. Based on the standard Yager T-norm, an adaptive strategy is designed for its index parameter, and then, the nondifferentiability property is eliminated. The antecedents of a fuzzy system, thus, can be optimized using the gradient-based method. CEMF is derived based on an exponential function and a traditional Gaussian membership function (GMF). Its lower bound is a positive value instead of 0, which is designed to match the use of adaptive Yager T-norm and, to some extent, enhances the performance of the AYATSK fuzzy system. The proposed approaches accord with the theoretical basis of fuzzy logic reasoning and, hence, the system interpretability is not reduced by adaptive Yager T-norm or CEMF. Experiments are conducted on both low-and high-dimensional classification datasets, whose feature dimensions vary from 8 to 120 432. This may be the first time for fuzzy systems to solve tasks with such a large range of feature dimensions. The experimental results demonstrate the superiority of AYATSK by comparing it with some state-of-the-art fuzzy algorithms.

Federated learning (FL) in Industrial IoT (IIoT) facilitates collaborative model training across distributed edge devices, ensuring data privacy and localized insights without centralized data aggregation. However, the networked parameter sharing mechanism in FL renders it vulnerable to exploitation by man-in-the-middle (MITM) attackers, potentially disrupting the model training process. To mitigate this threat, this article presents a novel blockchain-reinforced FL architecture aimed at enabling cooperative intrusion detection. Initially, FL is leveraged to aggregate all learned information from edge servers, thereby disseminating extracted attack characteristics to all participants through gradient sharing. Subsequently, a blockchain-based parameter verification scheme is introduced to safeguard against tampered local parameters affecting the global model. Clients record model parameters in smart contracts deployed on a private chain, and parameter servers verify parameter confidentiality before aggregation, ensuring only valid parameters are considered. Finally, extensive experiments are conducted using an edge IIoT cybersecurity data set comprising 61 features spanning ten protocol layers and five attacks targeting IIoT connectivity protocols. Simulation results demonstrate that the proposed scheme significantly enhances intrusion detection accuracy, achieving a threefold improvement when two-thirds of federated nodes are subjected to MITM attacks.

Approximation ability is of much importance for neural networks. The broad learning system (BLS) (Chen and Liu, 2018), widely used in the industry with good performance, has been proved to be a universal approximator from the aspect of density. This kind of approximation property is very important, which proves the existence of the desired network but does not provide a means of construction that is commonly implemented through complexity aspect. Thus, such an approach lacks the advantage of determining constructively the network architecture and its weights. To the best of our knowledge, for a BLS, there is a few theory providing a constructive approach to obtain the network structure along with weights ensuring the approximation properties. By virtue of the long-term memory and nonlocality properties, fractional calculus has observed many distinctive applications. The purpose of this article is to study the BLS approximation ability constructively, which is valid for fractional case as well. Specifically, first we introduce two simplified BLSs by means of extending functions. For each of the simplified BLSs, an upper bound of error is derived through the modulus of continuity of Caputo fractional derivatives. As a result, two types of fractional convergent behaviors of BLS, that is: 1) pointwise and 2) uniform convergence, have been rigorously proved as well. Finally, some numerical experiments are conducted to demonstrate the approximation capabilities of BLSs.

Multi-Label Classification (MLC) refers to associating each instance with multiple labels simultaneously. MLC has gained much importance due to its ability to better reflect the complexity of the real world classification problems. Fuzzy System (FS) has excellent nonlinear modeling capability and strong interpretability, which makes it a promising model for complex MLC problems. However, it is widely known that FS suffers from the “curse of dimensionality”. Here, an adaptive membership function (MF) along with its generalized version are proposed to address high-dimensional problems. These MFs can effectively overcome “numeric underflow” in FS while preserving interpretability as much as possible.On this basis, a novel fuzzy rule based multi-label classification framework called Multi-Label High-Dimensional Takagi-Sugeno-Kang Fuzzy System (ML-HDTSK FS) is proposed. This model can handle data with over ten thousand dimensionality. Additionally, MLHDTSK FS uses a decomposed label correlation learning strategy to efficiently capture both high and low levels of relationship between labels, and adopts a group L21penalty to realize the learning of label-specific features. Combining these two new multi-label learning strategies and the novel adaptive membership function, ML-HDTSK FS becomes a more powerful tool for various MLC problems. The effectiveness of ML-HDTSK FS is demonstrated on seventeen benchmark multi-label data sets, and its performance is compared with eleven MLC algorithms. The experimental results confirm the validity of the proposed MLHDTSK FS, and demonstrate the superiority of it in dealing with MLC problems, especially for high dimensional ones.

Message passing interface (MPI) is a powerful tool for parallel computing, originally designed for high-performance computing on massively parallel computers. In this paper, we combine FERPSO-IMPR (fitness Euclidean distance ratio particle swarm optimizer with information migration-based penalty and population reshaping) and surrogate-assisted models to generate test cases for MPI program path coverage testing. In our proposed method, FERPSO-IMPR employs a dual population strategy to initialize data and calculate fitness. Then, we create a sample set based on the initial data and its fitness. Subsequently, we train the master-slave surrogate models to predict individual fitness. Finally, a small number of elite individuals are selected to execute the program to decide whether to generate the required test data and guide the subsequent evolution process. We apply the proposed method to seven MPI programs and perform experimental comparisons from five directions. Experimental results show that compared with the comparative method, the time consumption of the proposed method is reduced by 33.2%, the number of evaluations is reduced by 38.8%, and the success rate is increased by 7.6%. These results prove that our method can effectively reduce the test data generation cost of MPI programs.

Coping with imbalanced data is a challenging task in practical classification problems. One of effective methods to solve imbalanced problems is to oversample the minority class. Gls SMOTE is a classical oversampling method. However, it exhibits two disadvantages, namely, a linear generation and overgeneralization. In this article, an improved synthetic minority oversampling technique (SMOTE) method, FE- SMOTE, is proposed based on the idea of the method of finite elements. FE- SMOTE not only overcomes the above two disadvantages of SMOTE but also can generate samples that are more in line with the density distribution of the original minority class than those generated by the existing SMOTE variants. The originality of the proposed method stems from constructing a simplex for every minority sample and then triangulating it to expand the region of synthetic samples from lines to space. A new definition of the relative size for triangular elements not only helps determine the number of synthetic samples but also weakens the adverse impact of outliers. Generated samples by FE- SMOTE can effectively reflect the local potential distribution structure arising around every minority sample. Compared with 16 commonly studied oversampling methods, FE- SMOTE produces promising results quantified in terms of$G$-mean, AUC,$F$-measure, and accuracy on 22 benchmark imbalanced datasets and the big dataset MNIST.

Fuzzy inference systems (FISs) have been developed for many years but the use of FISs for high-dimensional problems is still a challenging task. The most frequently used T-norms for computing the firing strengths are product and minimum operators of which the former is often preferred because of its differentiability. However, for high-dimensional problems, the product T-norm suffers from the numeric underflow problem. Here, we primarily focus on addressing the problem that is associated with the use of the T-norms for designing high-dimensional FISs (HDFISs). For the product T-norm, we construct an HDFIS named HDFIS-prod, which easily escapes from the numeric underflow problem. The main novelty is that we propose an adaptive dimension-dependent membership function (DMF). For the minimum T-norm, an empirical observation led us to develop a mechanism that has the natural ability to deal with super high-dimensional problems, which results in another HDFIS named HDFIS-min. Both HDFIS-prod and HDFIS-min are tested on 18 datasets with feature dimensions varying from 1024 to 120450. The simulation results demonstrate that both of them have competitive performance on handling high-dimensional datasets.

As electric vehicle (EV) ownership becomes more commonplace, partly due to government incentives, there is a need also to design solutions such as energy allocation strategies to more effectively support sustainable vehicle-to-grid (V2G) applications. Therefore, this work proposes an energy allocation strategy, designed to minimize the electricity cost while improving the operating revenue. Specifically, V2G is abstracted as a three-domain network architecture to facilitate flexible, intelligent, and scalable energy allocation decision-making. Furthermore, this work combines virtual network embedding (VNE) and deep reinforcement learning (DRL) algorithms, where a DRL-based agent model is proposed, to adaptively perceives environmental features and extracts the feature matrix as input. In particular, the agent consists of a four-layer architecture for node and link embedding, and jointly optimizes the decision-making through a reward mechanism and gradient back-propagation. Finally, the effectiveness of the proposed strategy is demonstrated through simulation case studies. Specifically, compared to the used benchmarks, it improves the VNR acceptance ratio, Long-term average revenue, and Long-term average revenue-cost ratio indicators by an average of 3.17%, 191.36, and 2.04%, respectively. To the best of our knowledge, this is one of the first attempts combining VNE and DRL to provide an energy allocation strategy for V2G.

This letter focuses on the robust low-rank matrix recovery (RLRMR) in the presence of gross sparse outliers. Instead of using$\ell _{1}$-norm to reduce or suppress the influence of anomalies, we aim to eliminate their impact. To this end, we model the RLRMR as a mixed integer programming (MIP) problem based on the$\ell _{0}$-norm. Then, a block coordinate descent (BCD) algorithm is developed to iteratively solve the resultant MIP. At each iteration, the proposed approach first utilizes the$\ell _{0}$-norm optimization theory to assign binary weights to all entries of the residual between the known and estimated matrices. With these binary weights, the optimization over the bilinear term is reduced to a weighted extension of the Frobenius norm. As a result, the optimization problem is decomposed into a group of row-wise and column-wise subproblems with closed-form solutions. Additionally, the convergence of the proposed algorithm is studied. Simulation results demonstrate that the proposed method is superior to five state-of-the-art RLRMR algorithms.

A major limitation of fuzzy or neuro-fuzzy systems is their failure to deal with high-dimensional datasets. This happens primarily due to the use of T-norm, particularly, product or minimum (or a softer version of it). Thus, there are hardly any work dealing with datasets having features more than hundred or so. Here, we propose a neuro-fuzzy framework that can handle datasets with even more than 7000 features! In this context, we propose an adaptive softmin (Ada-softmin) which effectively overcomes the drawbacks of “numeric underflow” and “fake minimum” that arise for existing fuzzy systems while dealing with high-dimensional problems. We call it an adaptive Takagi–Sugeno–Kang (AdaTSK) fuzzy system. We then equip the AdaTSK system to perform feature selection and rule extraction in an integrated manner. In this context, a novel gate function is introduced and embedded only in the consequent parts, which can determine the useful features and rules, in two successive phases of learning. Unlike conventional fuzzy rule bases, we design an enhanced fuzzy rule base, which maintains adequate rules but does not grow the number of rules exponentially with features that typically happens for fuzzy neural networks. The integrated feature selection and rule extraction AdaTSK (FSRE-AdaTSK) system consists of three sequential phases: 1) feature selection; 2) rule extraction; and 3) fine tuning. The effectiveness of the FSRE-AdaTSK is demonstrated on 19 datasets of which five are in more than 2000 dimension including two with more than 7000 features. This may be the first attempt to develop fuzzy rule-based classifiers that can directly deal with more than 7000 features without requiring separate selection of features or any other dimensionality reduction method.

Fuzzy or neuro-fuzzy systems have been successfully employed in many areas, but their limitation in solving high-dimensional problems remains a challenging task. On the other hand, fuzzy model-based feature selection (FS) approaches have been well studied but, because of the above limitation, are rarely applied to high-dimensional data, which exactly needs feature reduction. In this article, we design a novel adaptive Ln-Exp softmin operator as the approximator of the minimum T-norm, which is equipped in Takagi–Sugeno–Kang (TSK) fuzzy systems to make them capable of dealing with high-dimensional problems. The adaptive Ln-Exp softmin-based TSK (ALETSK) model is developed. Then, we improve the gate function by introducing an enhanced scheme and propose a double groups of gates-based ALETSK (DG-ALETSK) fuzzy approach to simultaneously conduct FS and rule extraction (RE), where the gate function is a function measuring the importance of the features or rules and it acts like a gate. More specifically, a group of feature gates are embedded in the antecedents of ALETSK for FS. Meanwhile, another group of rule gates are embedded in the consequents of ALETSK for RE. In one training phase, the gate parameters are trained along with the system parameters, so that the FS and RE are done together. Our proposed DG-ALETSK is effective and time-saving for FS and RE in the high-dimensional tasks, which is verified by the numerical experiments.

Many video websites offer packages composed of paid videos. Users who purchase a package become members of the website, and thus can enjoy the membership service, such as watching the paid videos. It is practically important to predict which users will become members so that the website can recommend them the suitable packages for purchasing. Existing works generally predict the purchase behavior of users through capturing their interests in items. However, such works cannot be directly applied to the studied problem due to the following challenges. First, some important features of videos and packages change over time, such as the number of clicks and the update of the videos. Existing methods are not capable to capture such dynamic features. Second, a user's purchasing intention is very hard to capture. A user watching a video does not necessarily mean that he/she would like to purchase the corresponding package. In this paper, we propose a Dynamic Evolution based Deep Hierarchical Intention Network (Dy-HIEN for short) for membership prediction, which contains two modules. In the first module, we design a dynamic embedding learning method, applying multi-relational heterogeneous information network and attention mechanism to effectively represent the embedding of videos and packages. In the second module, a hierarchical method is proposed to extract the purchase intention of users. First, the video play history is divided into sessions based on the clicks on packages, and then time-order encoder and kernel functions are applied to mine the intention pattern associated with the package clicked in each session. Extensive experiments on real-world datasets are conducted to demonstrate the advantages of the proposed model on a variety of evaluation metrics.

Evolutionary multitasking (EMT) is a newly emerging research topic in the community of evolutionary computation, which aims to improve the convergence characteristic across multiple distinct optimization tasks simultaneously by triggering knowledge transfer among them. Unfortunately, most of the existing EMT algorithms are only capable of boosting the optimization performance for homogeneous problems which explicitly share the same (or similar) fitness landscapes. Seldom efforts have been devoted to generalize the EMT for solving heterogeneous problems. A few preliminary studies employ domain adaptation techniques to enhance the transferability between two distinct tasks. However, almost all of these methods encounter a severe issue which is the so-called degradation of intertask mapping. Keeping this in mind, a novel rank loss function for acquiring a superior intertask mapping is proposed in this article. In particular, with an evolutionary-path-based representation model for optimization instance, an analytical solution of affine transformation for bridging the gap between two distinct problems is mathematically derived from the proposed rank loss function. It is worth mentioning that the proposed mapping-based transferability enhancement technique can be seamlessly embedded into an EMT paradigm. Finally, the efficacy of our proposed method against several state-of-the-art EMTs is verified experimentally on a number of synthetic multitasking and many-tasking benchmark problems, as well as a practical case study.

Sparse multiobjective optimization problems (MOPs) have become increasingly important in many applications in recent years, e.g., the search for lightweight deep neural networks and high-dimensional feature selection. However, little attention has been paid to sparse large-scale MOPs, whose Pareto-optimal sets are sparse, i.e., with many decision variables equal to zero. To address this issue, this article proposes an enhanced competitive swarm optimization algorithm assisted by a strongly convex sparse operator (SCSparse). A tricompetition mechanism is introduced into competitive swarm optimization, aiming to strike a better balance between exploration and exploitation. In addition, the SCSparse is embedded in the position updating of the particles to generate sparse solutions. Our simulation results show that the proposed algorithm outperforms the state-of-the-art methods on both sparse test problems and application examples.

Complex-valued neural network is a kind of learning model which can deal with problems in complex domain. Fully complex extreme learning machine (CELM) is a much faster training algorithm than the complex backpropagation (CBP) scheme. However, it is at the cost of using more hidden nodes to obtain the comparable performance. An upper-layer-solution-aware algorithm has been proposed for training single-hidden layer feedforward neural networks, which performs much better than its counterparts, pseudo-inverse learning (PIL)/extreme learning machine and gradient decent-based backpropagation neural networks. Consequently, there exist two challenges that need to be dealt with: 1) How to combine the advantages of CBP and CELM to develop a novel complex learning algorithm? and 2) What is the convergent behavior of the presented algorithm? In this article, an input weights dependent complex-valued (IWDCV) learning algorithm based on Wirtinger calculus has been proposed, which effectively solves the nonanalytic problem of the common activation functions during training neural networks. In addition, the monotonicity of the error function and the deterministic convergence of the proposed model have been strictly proved, which theoretically guarantee the efficiency and effectiveness of the given model, IWDCV. Finally, for real and complex-valued problems, a variety of simulations have been done to demonstrate the comparable performance of the proposed algorithm which support the theoretical observations as well.

-norm of weight matrix between the input and hidden layers. These penalty terms are nonsmooth at the origin, and hence, one simple but efficient smoothing technique is employed to overcome this issue. The monotonicity and convergence of the proposed algorithm are specified and proved under suitable assumptions. Then, extensive experiments are conducted on both artificial and real data sets. Empirical results explicitly demonstrate the ability of the proposed FS scheme and its effectiveness in controlling redundancy. The empirical simulations are observed to be consistent with the theoretical results.

Summary Steganalysis is to detect whether or not the seemly innocent image hiding message. It is an important research topic in information security. With the development of steganography technology, steganalysis becomes more and more difficult. Some steganalysis methods have been proposed to improve the performance. Most research work concentrates on special steganography information detection and the image steganography features are designed manually. Few research works concentrate on universal steganalysis methods. In this paper, as the first several attempts, a novel image steganalysis method based on deep neural network is proposed. First, image high‐frequency features are extracted with wavelet transformation method because that most image hiding message are high frequency. Second, high‐dimensional image steganography features are extracted with deep neural networks according to the high‐frequency images and informative features combination is selected with a novel feature selection method based on entropy. Then, a parallel SVM model is proposed to build the steganalysis model based on large scale training samples. At last, the efficiency of the proposed method is illustrated through analyzing a practical image steganalysis example.

We introduce the fractional-order global optimal backpropagation machine, which is trained by an improved fractional-order steepest descent method (FSDM). This is a fractional-order backpropagation neural network (FBPNN), a state-of-the-art fractional-order branch of the family of backpropagation neural networks (BPNNs), different from the majority of the previous classic first-order BPNNs which are trained by the traditional first-order steepest descent method. The reverse incremental search of the proposed FBPNN is in the negative directions of the approximate fractional-order partial derivatives of the square error. First, the theoretical concept of an FBPNN trained by an improved FSDM is described mathematically. Then, the mathematical proof of fractional-order global optimal convergence, an assumption of the structure, and fractional-order multi-scale global optimization of the FBPNN are analyzed in detail. Finally, we perform three (types of) experiments to compare the performances of an FBPNN and a classic first-order BPNN, i.e., example function approximation, fractional-order multi-scale global optimization, and comparison of global search and error fitting abilities with real data. The higher optimal search ability of an FBPNN to determine the global optimal solution is the major advantage that makes the FBPNN superior to a classic first-order BPNN.

We propose three different methods to determine the optimal number of hidden nodes based on L1regularization for a multilayer perceptron network. The first two methods, respectively, use a set of multiplier functions and multipliers for the hidden-layer nodes and implement the L1regularization on those, while the third method equipped with the same multipliers uses a smoothing approximation of the L1regularization. Each of these methods begins with a given number of hidden nodes, then the network is trained to obtain an optimal architecture discarding redundant hidden nodes using the multiplier functions or multipliers. A simple and generic method, namely, the matrix-based convergence proving method (MCPM), is introduced to prove the weak and strong convergence of the presented smoothing algorithms. The performance of the three pruning methods has been tested on 11 different classification datasets. The results demonstrate the efficient pruning abilities and competitive generalization by the proposed methods. The theoretical results are also validated by the results.

We propose an embedded/integrated feature selection method based on neural networks with Group Lasso penalty. Group Lasso regularization is considered to produce sparsity on the inputs to the network, i.e., for selection of useful features. Lasso based feature selection using a multi-layer perceptron usually requires an additional set of weights, while our Group Lasso formulation does not require that. However, Group Lasso penalty is non-differentiable at the origin. This may lead to oscillations in numerical simulations and make it difficult to analyze theoretically. To address this issue, four smoothing Group Lasso penalties are introduced. A rigorous proof for the convergence of the proposed algorithm is presented under suitable assumptions. To verify the effectiveness, a three-step algorithmic architecture is adopted in implementation. Experimental results on several datasets validate the theoretical results and demonstrate the competitive performance of the proposed method.

The application and theoretical analysis of fault tolerant learning are very important for neural networks. Our objective here is to realize fault tolerant sparse multilayer perceptron (MLP) networks. The stochastic gradient descent method has been employed to perform online learning for MLPs. For weight noise injection-based network models, it is a common strategy to add a weight decay regularizer while constructing the objective function for learning. However, this${l}_{2}$-norm penalty does not generate sparse optimal solutions. In this paper, a group lasso penalty term is used as a regularizer, where a group is defined by the set of weights connected to a node from nodes in the preceding layer. Group lasso penalty enables us to prune redundant hidden nodes. Due to its nondifferentiability at the origin, a smooth approximation of the group lasso penalty is developed. Then, a rigorous proof for the asymptotic convergence of the learning algorithm is provided. Finally, some simulations have been performed to verify the sparseness of the network and the theoretical results.

In this paper, we propose four new variants of the backpropagation algorithm to improve the generalization ability for feedforward neural networks. The basic idea of these methods stems from the Group Lasso concept which deals with the variable selection problem at the group level. There are two main drawbacks when the Group Lasso penalty has been directly employed during network training. They are numerical oscillations and theoretical challenges in computing the gradients at the origin. To overcome these obstacles, smoothing functions have then been introduced by approximating the Group Lasso penalty. Numerical experiments for classification and regression problems demonstrate that the proposed algorithms perform better than the other three classical penalization methods, Weight Decay, Weight Elimination, and Approximate Smoother, on both generalization and pruning efficiency. In addition, detailed simulations based on a specific data set have been performed to compare with some other common pruning strategies, which verify the advantages of the proposed algorithm. The pruning abilities of the proposed strategy have been investigated in detail for a relatively large data set, MNIST, in terms of various smoothing approximation cases.

Compared with traditional learning methods such as the back propagation (BP) method, extreme learning machine provides much faster learning speed and needs less human intervention, and thus has been widely used. In this paper we combine the L 1/2 regularization method with extreme learning machine to prune extreme learning machine. A variable learning coefficient is employed to prevent too large a learning increment. A numerical experiment demonstrates that a network pruned L 1/2 regularization has fewer hidden nodes but provides better performance than both the original network and the network pruned by L 2 regularization.

Weight-decay method as one of classical complexity regularizations is simple and appears to work well in some applications for multi-layer perceptron network (MPN). This paper shows results for the weak and strong convergence for cyclic and almost cyclic learning MPN with penalty term (weight-decay). The convergence is guaranteed under some relaxed conditions such as the activation functions, learning rate and the assumption for the stationary set of error function. Furthermore, the boundedness of the weights in the training procedure is obtained in a simple and clear way.

Two backpropagation algorithms with momentum for feedforward neural networks with a single hidden layer are considered. It is assumed that the training samples are supplied to the network in a cyclic or an almost-cyclic fashion in the learning procedure, i.e., in each training cycle, each sample of the training set is supplied in a fixed or a stochastic order respectively to the network exactly once. A restart strategy for the momentum is adopted such that the momentum coefficient is set to zero at the beginning of each training cycle. Corresponding weak and strong convergence results are then proved, indicating that the gradient of the error function goes to zero and the weight sequence goes to a fixed point, respectively. The convergence conditions on the learning rate, the momentum coefficient, and the activation functions are much relaxed compared with those of the existing results.