Advancement of Neural Architecture Search (NAS) has the potential to significantly improve the efficiency and performance of machine learning systems, as well as enable the exploration of new architectures and applications across a wide range of fields. A promising direction for developing more scalable and adaptive neural network architectures is the Once-for-All (OFA), a NAS framework that decouples the training and the search stages, meaning that one super-network is trained once, and then multiple searches can be performed according to different deployment scenarios. More recently, the OFA2strategy improved the search stage of the OFA framework by taking advantage of the very low cost of sampling already trained sub-networks and by exploring the multi-objective nature of the problem: a set of non-dominated sub-networks are all obtained at once, with distinct trade-offs involving hardware constraints and accuracy. In this work, we propose OFA3, building high-performance ensembles by solving the problem of how to automatically select the optimal subset of the already obtained non-dominated sub-networks. Particularly when components of the ensemble can run in parallel, our results dominate any other configuration of the available sub-networks, taking accuracy and latency as the conflicting objectives. The source code is available at https://github.com/ito-rafael/once-for-all-3.

Once-for-All (OFA) is a Neural Architecture Search (NAS) framework designed to address the problem of searching efficient architectures for devices with different resources constraints by decoupling the training and the searching stages. The computationally expensive process of training the OFA neural network is done only once, and then it is possible to perform multiple searches for subnetworks extracted from this trained network according to each deployment scenario. In this work we aim to give one step further in the search for efficiency by explicitly conceiving the search stage as a multi-objective optimization problem. A Pareto frontier is then populated with efficient, and already trained, neural architectures exhibiting distinct trade-offs among the conflicting objectives. This could be achieved by using any multi-objective evolutionary algorithm during the search stage, such as NSGA-II and SMS-EMOA. In other words, the neural network is trained once, the searching for subnetworks considering different hardware constraints is also done one single time, and then the user can choose a suitable neural network according to each deployment scenario. The conjugation of OFA and an explicit algorithm for multi-objective optimization opens the possibility of a posteriori decision-making in NAS, after sampling efficient subnetworks which are a very good approximation of the Pareto frontier, given that those subnetworks are already trained and ready to use. The source code is available at https://github.com/ito-rafael/once-for-all-2.

This work aimed to develop a personalized multi-objective recommender system for food diets, which seeks to suggest to the user a diverse list of four meals a day (breakfast, lunch, snack, dinner). The efficient solutions are capable of simultaneously meeting a list of nutritional specifications and also minimizing both the concentration in a specific meal or food item and the total cost of acquisition and preparation. Efficient solutions are sought from the joint use of the NSGA-II and Gurobi optimization packages, after formulating the diet problem as a bilevel optimization: a combinatorial and multi-objective problem at the upper level – which food items from each category (the available food categories and the categories allocated to each of the four meals are defined in advance) should make up the diet – and a mathematical programming problem at the lower level – what is the optimal amount of the selected food items to compose the diet, given nutritional constraints. As Gurobi does not operate directly with a multi-objective optimization perspective, its lower-level objective function involves maximizing the total energy of the daily diet. Experimental results, considering fictitious food costs, show that NSGA-II and Gurobi operate in synergy, providing a diverse list of menus for the four daily meals, thus making a valuable approximation of the Pareto frontier. The distinctive aspect of this multi-objective bilevel solution to the diet problem, then, resides in the supply of diverse and, at the same time, efficient candidate solutions, in the sense of achieving the Pareto frontier and being scattered along its extension.

This paper extends the recently conceived learning mechanism called EVeP (Extreme Value evolving Predictor), an evolving fuzzy-rule-based predictor characterized by innovative procedures to define the antecedent and consequent parts of the existing fuzzy rules. In EVeP, information granules are recursively updated and associated with Weibull distributions, a generalization of Gaussian distributions which incorporates more robust statistics to establish the region of influence of each fuzzy rule. Shared information from all the rules, in a multitask formulation, is adopted to set the consequent parameters in EVeP. Given that the multi-task formulation is solved using batch learning and gradient descent, the computational cost per iteration tends to be high, being a concern in practical applications. Therefore, here the multi-task framework at the consequent part of the rules was revised to incorporate online convex optimization, given rise to EVeP_OCO. Now, antecedent and consequent parts of the rules are updated in a fully recursive way, with a clear reduction in the computational burden per iteration, particularly when the worst case scenarios are considered: the cost per iteration depends on the current number of rules to be updated. The case studies are composed of a variety of benchmark time series prediction problems. They demonstrate the significant gain in terms of computational cost per iteration, with an admissible reduction in performance by replacing a batch multi-task learning procedure by an online counterpart.

This article introduces a new evolving fuzzy-rule-based algorithm for online data streams, named extreme value evolving predictor (EVeP). It offers a statistically well-founded approach to define the evolving fuzzy granules that form the antecedent and the consequent parts of the rules. The evolving fuzzy granules correspond to radial inclusion Weibull functions. They are interpreted by the extreme value theory as the limiting distribution of the relative proximity among the rules of the learning model. Regarding the parameters of the Takagi–Sugeno term at the consequent of the rules, the algorithm enhances the already demonstrated benefits of multitask learning by replacing a binary version with a fuzzy structural relationship among the rules. The pairwise similarity among the rules is automatically provided by the current interaction of the evolving fuzzy granules at the antecedent and at the consequent parts of their corresponding rules. Several computational experiments, using artificial and real-world time series, attest to the dominating prediction performance of EVeP when compared to the state-of-the-art evolving algorithms.

In this article, we present the Group Asymmetric Multi-Task Learning (GAMTL) algorithm that automatically learns from data how tasks transfer information among themselves at the level of a subset of features. In practice, for each group of features GAMTL extracts an asymmetric relationship supported by the tasks, instead of assuming a single structure for all features. The additional flexibility promoted by local transference in GAMTL allows any two tasks to have multiple asymmetric relationships. The proposed method leverages the information present in these multiple structures to bias the training of individual tasks towards more generalizable models. The solution to the GAMTL’s associated optimization problem is an alternating minimization procedure involving tasks parameters and multiple asymmetric relationships, thus guiding to convex smaller sub-problems. GAMTL was evaluated on both synthetic and real datasets. To evidence GAMTL versatility, we generated a synthetic scenario characterized by diverse profiles of structural relationships among tasks. GAMTL was also applied to the problem of Alzheimer’s Disease (AD) progression prediction. Our experiments indicated that the proposed approach not only increased prediction performance, but also estimated scientifically grounded relationships among multiple cognitive scores, taken here as multiple regression tasks, and regions of interest in the brain, directly associated here with groups of features. We also employed stability selection analysis to investigate GAMTL’s robustness to data sampling rate and hyper-parameter configuration. GAMTL source code is available on GitHub: https://github.com/shgo/gamtl .

Symbolic regression aims to hypothesize a functional relationship involving explanatory variables and one or more dependent variables, based on examples of the desired input-output behavior. Genetic programming is a meta-heuristic commonly used in the literature to achieve this goal. Even though Symbolic Regression is sometimes associated with the potential of generating interpretable expressions, there is no guarantee that the returned function will not contain complicated constructs or even bloat. The Interaction-Transformation (IT) representation was recently proposed to alleviate this issue by constraining the search space to expressions following a simple and comprehensive pattern. In this paper, we resort to Simulated Annealing to search for a symbolic expression using the IT representation. Simulated Annealing exhibits an intrinsic ability to escape from poor local minima, which is demonstrated here to yield competitive results, particularly in terms of generalization, when compared with state-of-the-art Symbolic Regression techniques, that depend on population-based meta-heuristics, and committees of learning machines.

The viability of wind power massive use goes through an effective estimation of the power to be produced in wind farms, one of the fastest growing sources of renewable energy. Therefore, short-term wind speed forecasting involving several turbines and possibly multiple wind farms becomes a crucial practical demand. Once there are historical time series of wind speed intensity associated with subgroups of turbines, multiple tasks of time series prediction should be solved simultaneously. Aiming at performance improvement, we propose here the adoption of information sharing in multi-task learning frameworks, involving linear and nonlinear prediction models, and time series with and without differentiation. The nonlinear multi-task learning approach is based on extreme learning machines (ELMs) and guided to the best accuracies on a real-world case study of short-term wind speed forecasting. A suitable tuning of the variance in the distribution of the random hidden layer weights was demonstrated to be an essential preprocessing step for the ELMs, together with regularization routines. Additionally, it was found that properly incorporating tasks relationship into the learning process can systematically contribute to better generalization, particularly when compared to the single task learning counterparts.

This paper introduces two novel contributions to the online learning algorithm called Fuzzy set Based evolving Modeling with Multitask Learning (FBeM_MTL), the first algorithm in the literature to consider multitask learning in the context of data stream, adaptive and evolving systems. In this new version, the degree of intersection of the information granules is directly used to define a real-valued matrix representing the relationship among the learning tasks, responsible for defining the parameters of the consequent part of all functional IF-THEN fuzzy rules. Unlike the original FBeM_MTL, in this new version, we eliminated the need for the binarization of the matrix representing the connected rules, guiding to both performance improvement and reduction in the number of user-defined parameters. The second contribution is the adoption of the Weighted Least Squares (WLS) method to define the parameters of the consequent part of the rules, using the similarity measure between every pair of samples to the mean point to set their corresponding weights in the WLS problem. Computational experiments on time series prediction of weather temperature, rain precipitation, wind speed in eolian farms and stock exchange are used to validate the performance of this new version. When compared to the original FBeM_MTL and also to several other state-of-the-art evolving systems in the literature, our approach guides to competitive results using a reduced number of parameters.

On binary classification, the goal of minimizing the false positive and false negative rates creates a conflict, being impossible to optimize both simultaneously. This challenge is even more significant on imbalanced classification datasets since an incorrect choice of the relative relevance of each objective on the optimization can lead to ignoring, or poorly learning the minority class. The proposal of this work takes into account the existing conflict among the learning losses of the classes, and use a deterministic multi-objective optimization method, called MONISE, to create a set of solutions with diverse misclassification tradeoffs among the classes. Since accuracy is no longer a proper criterion for imbalanced datasets, we had to resort to multiple criteria to report the performance: each classifier, proposed or competitors, was selected and reported using the same metrics. We used F1, kappa and g-mean for general evaluation of performance and Fβs (F1/16, F1/4, F4and F16) to emulate a shifting decision maker preference from precision to recall; all comparisons were made using a Friedman test with Finner posthoc test. However, when we take into account multiple metrics without any prior knowledge, it may become impossible to pinpoint the best method, since the evaluation criteria may also be in conflict. Again, to solve this, we resorted to a Friedman test with a non-dominated ranking. With this multi-criteria analyses, we conclude that explicitly considering multiple objectives on the optimization can guide to promising results.

Microarrays can be employed to better characterise allergies, as interactions between antibodies and allergens in mammals can be monitored. Once the joint dynamics of these elements in both healthy and diseased animals are understood, a model to predict the likelihood of an individual having allergic reactions can be defined. We investigate the potential use of Dynamic Selection (DS) methods to classify protein microarray data, with a case study of equine insect bite hypersensitivity (IBH) disease. To the best of our knowledge DS has not yet been applied to these data types. Since most microarrays datasets have a low number of samples, we hypothesise that DS models will produce satisfactory results due to their ability to perform better when compared to traditional ensemble techniques for similar data. We focus on three research questions: 1) What is the potential of DS for microarray data classification and how does it compare with existing classical classification methods results? 2) how do DS methods perform for the IBH dataset? and 3) does feature selection improve DS performance for this data? A wrapper using backward elimination and embedded with a regularized extreme learning machine are adopted to identify the more relevant features influencing the onset of the disease. Results from traditional classifiers are compared to 21 different DS methods before and after performing feature selection. Our results indicate that DS methods do not outperform single and static classifiers on this high-dimensional dataset and their performance also does not improved after feature selection.

Group LASSO is a widely used regularization that imposes sparsity considering groups of covariates. When used in Multi-Task Learning (MTL) formulations, it makes an underlying assumption that if one group of covariates is not relevant for one or a few tasks, it is also not relevant for all tasks, thus implicitly assuming that all tasks are related. This implication can easily lead to negative transfer if this assumption does not hold for all tasks. Since for most practical applications we hardly know a priori how the tasks are related, several approaches have been conceived in the literature to (i) properly capture the transference structure, (ii) improve interpretability of the tasks interplay, and (iii) penalize potential negative transfer. Recently, the automatic estimation of asymmetric structures inside the learning process was capable of effectively avoiding negative transfer. Our proposal is the first attempt in the literature to conceive a Group LASSO with asymmetric transference formulation, looking for the best of both worlds in a framework that admits the overlap of groups. The resulting optimization problem is solved by an alternating procedure with fast methods. We performed experiments using synthetic and real datasets to compare our proposal with state-of-the-art approaches, evidencing the promising predictive performance and distinguished interpretability of our proposal. The real case study involves the prediction of cognitive scores for Alzheimer's disease progression assessment. The source codes are available at GitHub.

Genetic Programming is a branch of Evolutionary Computation devoted to the evolution of programs. Several genetic operators have been proposed to increase the power of the search, given that the space of admissible programs is very challenging to be properly explored toward high quality solutions. Semantically-driven genetic operators are gaining more attention lately, given that the behaviour of the search operators are more predictable, possibly leading to a more efficient evolution. Nonetheless, Semantic Genetic Programming may undergo the bloat phenomenon, characterized by an uncontrolled increase in the program size along the generations. Some attempts have been made in the literature to refrain code bloat, and here we are proposing three novel semantic-driven mutation operators for the tree structure codification. A multi-objective perspective is adopted, where the mutated subtrees correspond to nondominated instances in a previously defined library of candidate subtrees. Several conflicting objectives may be incorporated into the decision making process, such as accuracy, semantic distance to a reference behaviour, and size of the subtree. Experimental results reveal that our proposed operators are effective in restraining bloating, without a negative impact on the other performance metrics, and are competitive with other relevant approaches available in the literature.

Regularized multitask learning is explicitly interpreted hereas a many-objective optimization problem, dealt with a deterministic solver that properly controls the sampling of the Pareto frontier. Each objective function corresponds to the learning loss of a task, so that we have as many objectives as tasks. The obtained Pareto-optimal models are then explored to implement distinct learning sharing strategies: (1) by considering a single parameter vector for all tasks, the simplest learning model that could have been conceived in multitask learning, the distinct trade-offs along the Pareto frontier can be interpreted as efficient and diverse sharing perspectives for the multiple tasks; (2) those distinct sharing perspectives are then aggregated in an ensemble or the best model in the validation set is selected. Notice that using a single parameter vector for all tasks in our many-objective perspective should not be directly associated with that naive, and generally of low performance, procedure of taking all tasks as being equally related. Distinct trade-offs automatically promote the proposition of efficient and structurally diverse relationships among the learning tasks, which support a competitive performance when compared with consolidated multitask learning methods in classification problems.

Future projection of climate is typically obtained by combining outputs from multiple Earth System Models (ESMs) for several climate variables such as temperature and precipitation. While IPCC has traditionally used a simple model output average, recent work has illustrated potential advantages of using a multitask learning (MTL) framework for projections of individual climate variables. In this paper we introduce a framework for hierarchical multitask learning (HMTL) with two levels of tasks such that each super-task, i.e., task at the top level, is itself a multitask learning problem over sub-tasks. For climate projections, each super-task focuses on projections of specific climate variables spatially using an MTL formulation. For the proposed HMTL approach, a group lasso regularization is added to couple parameters across the super-tasks, which in the climate context helps exploit relationships among the behavior of different climate variables at a given spatial location. We show that some recent works on MTL based on learning task dependency structures can be viewed as special cases of HMTL. Experiments on synthetic and real climate data show that HMTL produces better results than decoupled MTL methods applied separately on the super-tasks and HMTL significantly outperforms baselines for climate projection.

This paper introduces necessary and sufficient conditions that surrogate functions must satisfy to properly define frontiers of nondominated solutions in multiobjective optimization (MOO) problems. These new conditions work directly on the objective space, and thus are agnostic about how the solutions are evaluated. Therefore, real objectives or user-designed objectives' surrogates are allowed, opening the possibility of linking independent objective surrogates. To illustrate the practical consequences of adopting the proposed conditions, we use Gaussian processes (GPs) as surrogates endowed with monotonicity soft constraints and with an adjustable degree of flexibility, and compare them to regular GPs and to a frontier surrogate method in the literature that is the closest to the method proposed in this paper. Results show that the necessary and sufficient conditions proposed here are finely managed by the constrained GP, guiding to high-quality surrogates capable of suitably synthesizing an approximation to the Pareto frontier in challenging instances of MOO, while an existing approach that does not take the theory proposed in consideration defines surrogates which greatly violate the conditions to describe a valid frontier.

Multi-task learning (MTL) aims to improve generalization performance by learning multiple related tasks simultaneously. While sometimes the underlying task relationship structure is known, often the structure needs to be estimated from data at hand. In this paper, we present a novel family of models for MTL, applicable to regression and classification problems, capable of learning the structure of tasks relationship. In particular, we consider a joint estimation problem of the tasks relationship structure and the individual task parameters, which is solved using alternating minimization. The task relationship revealed by structure learning is founded on recent advances in Gaussian graphical models endowed with sparse estimators of the precision (inverse covariance) matrix. An extension to include flexible Gaussian copula models that relaxes the Gaussian marginal assumption is also proposed. We illustrate the effectiveness of the proposed model on a variety of synthetic and benchmark data sets for regression and classification. We also consider the problem of combining Earth System Model (ESM) outputs for better projections of future climate, with focus on projections of temperature by combining ESMs in South and North America, and show that the proposed model outperforms several existing methods for the problem.

In several applications, a solution must be selected from a set of tradeoff alternatives for operating in dynamic and noisy environments. In this paper, such multicriteria decision process is handled by anticipating flexible options predicted to improve the decision maker future freedom of action. A methodology is then proposed for predicting tradeoff sets of maximal hypervolume, where a multiobjective metaheuristic was augmented with a Kalman filter and a dynamical Dirichlet model for tracking and predicting flexible solutions. The method identified decisions that were shown to improve the future hypervolume of tradeoff investment portfolio sets for out-of-sample stock data, when compared to a myopic strategy. Anticipating flexible portfolios was a superior strategy for smoother changing artificial and real-world scenarios, when compared to always implementing the decision of median risk and to randomly selecting a portfolio from the evolved anticipatory stochastic Pareto frontier, whereas the median choice strategy performed better for abruptly changing markets. Correlations between the portfolio compositions and future hypervolume were also observed.

Multi-task learning (MTL) aims to improve generalization performance by learning multiple related tasks simultaneously. While sometimes the underlying task relationship structure is known, often the structure needs to be estimated from data at hand. In this paper, we present a novel family of models for MTL, applicable to regression and classification problems, capable of learning the structure of task relationships. In particular, we consider a joint estimation problem of the task relationship structure and the individual task parameters, which is solved using alternating minimization. The task relationship structure learning component builds on recent advances in structure learning of Gaussian graphical models based on sparse estimators of the precision (inverse covariance) matrix. We illustrate the effectiveness of the proposed model on a variety of synthetic and benchmark datasets for regression and classification. We also consider the problem of combining climate model outputs for better projections of future climate, with focus on temperature in South America, and show that the proposed model outperforms several existing methods for the problem.

An anticipatory stochastic multi-objective model based on S-Metric maximization is proposed. The environment is assumed to be noisy and time-varying. This raises the question of how to incorporate anticipation in metaheuristics such that the Pareto optimal solutions can reflect the uncertainty about the subsequent environments. A principled anticipatory learning method for tracking the dynamics of the objective vectors is then proposed so that the estimated S-Metric contributions of each solution can integrate the underlying stochastic uncertainty. The proposal is assessed for minimum holding, cardinality constrained portfolio selection, using real-world stock data. Preliminary results suggest that, by taking into account the underlying uncertainty in the predictive knowledge provided by a Kalman filter, we were able to reduce the sum of squared errors prediction of the portfolios ex-post return and risk estimation in out-of-sample investment environments.

This paper proposes a regularized hypervolume (SMetric) selection algorithm. The proposal is used for incorporating stability and diversification in financial portfolios obtained by solving a temporal sequence of multi-objective Mean Variance Problems (MVP) on real-world stock data, for short to longterm rebalancing periods. We also propose the usage of robust statistics for estimating the parameters of the assets returns distribution so that we are able to test two variants (with and without regularization) on dynamic environments under different levels of instability. The results suggest that the maximum attaining Sharpe Ratio portfolios obtained for the original MVP without regularization are unstable, yielding high turnover rates, whereas solving the robust MVP with regularization mitigated turnover, providing more stable solutions for unseen, dynamic environments. Finally, we report an apparent conflict between stability in the objective space and in the decision space.

This paper analyzes the application of a multi-population Gaussian-based estimation of distribution algorithm equipped with a restarting strategy and mutation, named MGcEDA, to the problem of estimating the Direction of Arrival (DOA) of time-varying plane waves impinging on a uniform linear array of sensors. This problem requires the minimization of a dynamic cost function which is non-linear, non-quadratic, multimodal and variant with respect to the signal-to-noise ratio. Experiments showed that MGcEDA was able to quickly respond to changes in the source features in scenarios with different levels of noise and number of signals. Moreover, MGcEDA outperforms a previously proposed approach in all considered experiments in terms of well known performance measures.

Fuzzy Cognitive Map (FCM) is a tool for modeling and representing discrete dynamical systems. Several approaches were proposed for the automatic learning of FCM on the basis of historical data. The learning techniques can be grouped into three types: Hebbian-based, population-based, and hybrid, which combines both types. Despite the good overall results achieved by population-based approaches relative to the other learning paradigms, it is possible to improve their performance by combining them with local search procedures. In this paper, we investigate the performance of a multi-start gradient-based method and two evolutionary methods hybridized with a gradient-based local search procedure for the learning of FCMs. We tested the proposed approaches for synthetic and real world FCM models. The results show that it was possible to improve the performance of the evolutionary methods with a relatively small increase in the resultant computational time.

Echo state networks (ESNs) characterize an attractive alternative to conventional recurrent neural network (RNN) approaches as they offer the possibility of preserving, to a certain extent, the processing capability of a recurrent architecture and, at the same time, of simplifying the training process. However, the original ESN architecture cannot fully explore the potential of the RNN, given that only the second-order statistics of the signals are effectively used. In order to overcome this constraint, distinct proposals promote the use of a nonlinear readout aiming to explore higher-order available information though still maintaining a closed-form solution in the least-squares sense. In this work, we review two proposals of nonlinear readouts - a Volterra filter structure and an extreme learning machine - and analyze the performance of these architectures in the context of two relevant signal processing tasks: supervised channel equalization and chaotic time series prediction. The obtained results reveal that the nonlinear readout can be decisive in the process of aproximating the desired signal. Additionally, we discuss the possibility of combining both ideas of nonlinear readouts and preliminary results indicate that a performance improvement can be attained.

In this work, we present a novel framework for automatic feature selection in brain-computer interfaces (BCIs). The proposal, which manipulates features generated in the frequency domain by an estimate of the power spectral density of the EEG signals, is based on feature optimization (with both binary and real coding) using a state-of-the-art artificial immune network, the cob-aiNet. In order to analyze the performance of the proposed framework, two approaches are adopted: a direct use of the Davies-Bouldin index and the use of metrics associated with the operation of an extreme learning machine (ELM) in the role of a classifier. The results reveal that the proposal has the potential of improving the performance of a BCI system, and also provide elements for an analysis of the spectral content of EEG signals and of the performance of ELMs in motor imagery paradigms.

The class of inventory routing problems (IRPs) is present in several areas, including automotive industry and cash management for ATM networks. In the specific case of vendor-managed IRPs, in which the supplier is responsible for managing the product inventory in each client and for properly providing replenishments, the challenge is to determine which retailers should be served, the amount of product that should be delivered to each of these retailers, and which routes the distribution vehicles should follow, so that the associated costs are minimized. Although this is clearly a multi-objective optimization problem, in the literature it has been generally modeled as a single-objective problem, which limits the scope of the obtained results. Therefore, this work presents a multi-objective approach to solve one version of the IRP usually found in the scientific literature, by simultaneously minimizing both the inventory and transportation costs. The method proposed in this work is based on the well-known SPEA2 (Strength Pareto Evolutionary Algorithm) and includes innovative aspects mainly associated with the representation of candidate solutions, genetic operators and local search. The experiments were performed on a set of known benchmark IRPs from the literature, so that the obtained results could be properly compared to the best solution found for the single-objective version of each problem.

In this paper we apply an immune-inspired approach to train Multilayer Perceptrons (MLPs) for classification problems. Our proposal, called Gaussian Artificial Immune System (GAIS), is an estimation of distribution algorithm that replaces the traditional mutation and cloning operators with a probabilistic model, more specifically a Gaussian network, representing the joint distribution of promising solutions. Sub sequently, GAIS utilizes this probabilistic model for sampling new solutions. Thus, the algorithm takes into account the relationships among the variables of the problem, avoiding the disruption of already obtained high-quality partial solutions (building blocks). Besides the capability to identify and manipulate building blocks, the algorithm maintains diversity in the population, performs multimodal optimization and adjusts the size of the population automatically according to the problem. These attributes are generally absent from alternative algorithms, and all were shown to be useful attributes when optimizing the weights of MLPs, thus guiding to high-performance classifiers. GAIS was evaluated in six well-known classification problems and its performance compares favorably with that produced by contenders, such as opt-aiNet, IDEA and PSO.

Diversity maintenance is an important aspect in population-based metaheuristics for optimization, as it tends to allow a better exploration of the search space, thus reducing the susceptibility to local optima in multimodal optimization problems. In this context, metaheuristics based on the Artificial Immune System (AIS) framework, especially those inspired by the Immune Network theory, are known to be capable of stimulating the generation of diverse sets of solutions for a given problem, even though generally implementing very simple mechanisms to control the dynamics of the network. To increase such diversity maintenance capability even further, a new immune-inspired algorithm was recently proposed, which adopted a novel concentration-based model of immune network. This new algorithm, named cob-aiNet (Concentration-based Artificial Immune Network), was originally developed to solve real-parameter single-objective optimization problems, and it was later extended (with cob-aiNet[MO]) to deal with real-parameter multi-objective optimization. Given that both cob-aiNet and cob-aiNet[MO] obtained competitive results when compared to state-of-the-art algorithms for continuous optimization and also presented significantly improved diversity maintenance mechanisms, in this work the same concentration-based paradigm was further explored, in an extension of such algorithms to deal with single-objective combinatorial optimization problems. This new algorithm, named cob-aiNet[C], was evaluated here in a series of experiments based on four Traveling Salesman Problems (TSPs), in which it was verified not only the diversity maintenance capabilities of the algorithm, but also its overall optimization performance.

Biclustering is usually referred to as the process of finding subsets of rows and columns from a given dataset expressing a relationship. Each subset is a bicluster and corresponds to a sub-matrix whose elements tend to present a high degree of coherence with each other, that may lead to novel discoveries regarding the objects in the dataset. This coherence leads to the possibility of obtaining representative values for rows (subset of objects) and columns (subset of attributes) of each bicluster. In the literature, it is usually studied the additive coherence among elements, i.e. each element is represented by the sum of its respective representative values. But in a given dataset, it is also possible to find multiplicative relations, i.e. each element being represented by the multiplication of its respective representative values, and that may reveal distinct knowledge contained in the objects of the dataset. So, in this paper, a swarm based approach, named SwarmBcluster, is adapted to find both additive and multiplicative coherent biclusters from a dataset, in an attempt to enrich the amount of information provided by the biclusters. Experiments are performed considering two well known datasets and it is found that the multiplicative coherence biclusters improve the quality of the data analysis and may contribute to reduce the influence of noise.

In this paper, we propose an estimation of distribution algorithm based on an inexpensive Gaussian mixture model with online learning, which will be employed in dynamic optimization. Here, the mixture model stores a vector of sufficient statistics of the best solutions, which is subsequently used to obtain the parameters of the Gaussian components. This approach is able to incorporate into the current mixture model potentially relevant information of the previous and current iterations. The online nature of the proposal is desirable in the context of dynamic optimization, where prompt reaction to new scenarios should be promoted. To analyze the performance of our proposal, a set of dynamic optimization problems in continuous domains was considered with distinct levels of complexity, and the obtained results were compared to the results produced by other existing algorithms in the dynamic optimization literature.

There are two common strategies for technical loss reduction in electric power distribution networks: (a) the installation of capacitor banks to compensate the losses produced by reactive currents; and (b) the redefinition of the topology of electric distribution networks by changing the state of some sectionalizing switches to balance the load. Both strategies can be formulated as combinatorial optimization problems. The optimization problems for the first and the second strategies are usually known as Capacitor Placement Problem (CPP) and Network Reconfiguration Problem (NRP), respectively. In this paper, we propose a new approach based on Genetic Algorithm (GA) to solve both CPP and NRP simultaneously. The new approach makes use of two previously proposed and independent techniques for the CPP and the NRP. The performance of the new approach is compared with the performance of the two previously proposed techniques applied in a separate manner. The experiments show that the new method is more efficient regarding the metrics of power loss reduction and voltage profile enhancement.

In population-based meta-heuristics, the generation and maintenance of diversity seem to be crucial to deal with multimodal continuous optimization. However, usually this crucial aspect is not an inherent feature of generally adopted meta-heuristics. In this paper, we propose to associate diversity maintenance with the detection and elimination of redundant candidate solutions in the search space, more specifically candidate solutions located at the same attraction basin of a local optimum. Two low computational cost heuristics are proposed to detect redundancy, in a pairwise comparison of candidate solutions and by extracting local features of the fitness landscape at runtime. Those heuristics are not tied to a specific class of algorithms, and are thus able to be incorporated into a broad range of population-based meta-heuristics, and even into multiple executions of non-population-based algorithms. In a set of experimental results, the two heuristics were implemented as an attached module of an already existing multipopulation meta heuristics, and the results indicate that they operate properly, no matter the number and conformation of the attraction basins in multimodal optimization problems.

Although the missing data problem has been studied for many years, it is still a relevant and challenging problem nowadays. Data can be missing for a variety of reasons, and there are several techniques capable of processing missing data. A parcel of them tries to estimate the missing values. This technique is called imputation. Recently, it was proposed a biclustering algorithm, based on Swarm Intelligence, named SwarmBCluster, to impute missing data. As it is a novel and promising algorithm, this paper intends to investigate the influence of its parameters on the performance. To achieve this objective, this paper will compare SwarmBCluster with other two imputation algorithms and, after that, it will perform a sensitivity analysis. The quality of the imputations is measured with the Root Mean Squared Error (RMSE). The experiments showed that SwarmBCluster presents good results concerning the RMSE metric and that the proper choice of parameters can considerably improve the performance of the algorithm.

In this paper we apply an immune-inspired approach to generate fuzzy rule bases for classification problems. Our proposal, called Bayesian Artificial Immune System (BAIS), is a hybrid algorithm that replaces the traditional mutation and cloning operators with a probabilistic model, more specifically a Bayesian network, representing the joint distribution of promising solutions. Thus, the algorithm takes into account the relationships among the variables of the problem, avoiding the disruption of already obtained high-quality partial solutions (building blocks). Besides the capability to identify and manipulate building blocks, the algorithm maintains diversity in the population, performs multimodal optimization and adjusts the size of the population automatically according to the problem. These attributes are generally absent from alternative algorithms, and can be considered useful attributes when generating fuzzy rule bases, thus guiding to high-performance classifiers. BAIS was evaluated in six well-known classification problems and its performance compares favorably with that produced by contenders.

In this paper, the Reinforcement Learning problem is formulated equivalently to a Markov Decision Process. We address the solution of such problem using a novel Adaptive Dynamic Programming algorithm which is based on a Multilayer Perceptron Neural Network composed of a parameterized function approximator called Wire-Fitting. Extending such established model, this work makes use of concepts of eligibility to conceive faster learning algorithms. The advantage of the proposed approach is founded on the capability to handle continuous environments and to learn a better policy while following another. Simulation results involving the automatic control of an inverted pendulum are presented to indicate the effectiveness of the proposed algorithm.

Echo state networks represent a promising alternative to the classical approaches involving recurrent neural networks, as they ally processing capability, due to the existence of feedback loops within the dynamical reservoir, with a simplified training process. However, the existing networks cannot fully explore the potential of the underlying structure, since the outputs are computed via linear combinations of the internal states. In this work, we propose a novel architecture for an echo state network that employs the Volterra filter structure in the output layer together with the Principal Component Analysis technique. This idea not only improves the processing capability of the network, but also preserves the simplicity of the training process. The proposed architecture has been analyzed in the context of the channel equalization problem, and the obtained results highlight the adequacy and the advantages of the novel network, which achieved a convincing performance, overcoming the other echo state networks, especially in the most challenging scenarios.

In this paper, we apply an immune-inspired approach to design ensembles of heterogeneous neural networks for classification problems. Our proposal, called Bayesian artificial immune system, is an estimation of distribution algorithm that replaces the traditional mutation and cloning operators with a probabilistic model, more specifically a Bayesian network, representing the joint distribution of promising solutions. Among the additional attributes provided by the Bayesian framework inserted into an immune-inspired search algorithm are the automatic control of the population size along the search and the inherent ability to promote and preserve diversity among the candidate solutions. Both are attributes generally absent from alternative estimation of distribution algorithms, and both were shown to be useful attributes when implementing the generation and selection of components of the ensemble, thus leading to high-performance classifiers. Several aspects of the design are illustrated in practical applications, including a comparative analysis with other attempts to synthesize ensembles.

Metaheuristics based on the Artificial Immune System (AIS) framework, especially those inspired by the Immune Network theory, are known to be capable of stimulating the generation of diverse sets of solutions for a given problem, even though they generally implement very simple mechanisms to control the dynamics of the network. In the AIS literature, several studies propose different models that try to explain the behavior of immune networks, which are generally based on the concentration of antibodies and tend to better mimic some aspects of such complex systems. Therefore, in this work we propose a novel immune-inspired algorithm for optimization, named cob-aiNet (Concentration-based Artificial Immune Network), that intends to explore such network models and introduce new mechanisms to better control the dynamics of the network, so that a broader coverage of promising regions of the search space can be achieved. This property of cob-aiNet was verified in experimental analyses, in which the algorithm was compared to two other AIS proposals and also to all the competitors from the 2005 CEC Special Session on RealParameter Optimization.

Immune-inspired algorithms based on the Immune Network theory have been frequently claimed to be capable of maintaining diversity among the candidate solutions in their population. However, no specific study on this aspect to verify how the intrinsic diversity mechanisms of such immune algorithms behave, when compared to other approaches from the literature, has been made yet. Therefore, in this work we have addressed this issue, by taking the opt-aiNet algorithm (a popular immune-inspired algorithm developed for real-parameter optimization) and comparing its results with those of a modified version, in which the mechanisms associated with diversity maintenance were replaced by a traditional fitness sharing approach. Besides, two distance metrics were also considered for both algorithms: the traditional Euclidean distance, and the Line Distance, a metric proposed in the literature as capable of identifying whether two solutions belong to distinct local optima. The experiments were performed on six benchmark problems from the literature, each of them with distinct characteristics, and the results have shown that the original immune-inspired mechanisms of opt-aiNet are indeed more capable of stimulating the diversity of solutions, and also requiring a smaller amount of computational resources.

Biclustering is usually referred to as the process of finding subsets of rows and columns from a given dataset. Each subset is a bicluster and corresponds to a sub-matrix whose elements tend to present a high degree of coherence with each other. In order to find such structures, the δ-biclustering problem was formulated, being denoted as the problem of finding a set of biclusters limited by a maximum degree of coherence, measured by a mean-squared residue, while maximizing the bicluster total size. Additionally, it is expected a reduced overlap among the biclusters in the set, in other words, a minimization of the number of common elements shared by them. This also leads to a high coverage of the original dataset given the number of biclusters found. Most algorithms intended to find such biclusters focus only on the mean-squared residue and/or the bicluster size. This usually leads to a set of biclusters that do not fully cover the whole data and, as a consequence, shares a high overlap among them. This may generate redundant information on some portions of the dataset and lack of information on other portions. Also, some methods introduce noise into the dataset in order to promote a better coverage, but sometimes misleading the search. In this paper, a swarm-based approach, named SwarmBcluster, is created to effectively find biclusters without introducing noise and with the main objective of achieving maximum coverage. Experiments were performed considering two well-known datasets and a comparative analysis considering other approaches indicates that SwarmBcluster is capable of finding a set of biclusters with high coverage, while maintaining a high average volume and also obeying the coherence constraint imposed.

This paper proposes a Multi-Objective Gaussian Artificial Immune System (MOGAIS) to deal effectively with building blocks (high-quality partial solutions coded in the solution vector) in multi-objective continuous optimization problems. By replacing the mutation and cloning operators with a probabilistic model, more specifically a Gaussian network representing the joint distribution of promising solutions, MOGAIS takes into account the relationships among the variables of the problem, avoiding the disruption of already obtained high-quality partial solutions. The algorithm was applied to three benchmarks and the results were compared with those produced by state-of-the-art algorithms.

The biclustering technique was developed to avoid some of the drawbacks presented by standard clustering techniques. Given that biclustering requires the optimization of at least two conflicting objectives and that multiple independent solutions are desirable as the outcome, a few multi-objective evolutionary algorithms for biclustering were proposed in the literature. However, apart from the individual characteristics of the biclusters that should be optimized during their construction, several other global aspects should also be considered, such as the coverage of the dataset and the overlap among biclusters. These requirements will be addressed in this work with the MOM-aiNet+ algorithm, which is an improvement of the original multi-objective multipopulation artificial immune network denoted MOM-aiNet. Here, the MOM-aiNet+ algorithm will be described in detail, its main differences from the original MOM-aiNet will be highlighted, and both algorithms will be compared, together with three other proposals from the literature.

In many real-world scenarios, in contrast to standard benchmark optimization problems, we may face some uncertainties regarding the objective function. One source of these uncertainties is a constantly changing environment in which the optima change their location over time. New heuristics or adaptations to already available algorithms must be conceived in order to deal with such problems. Among the desirable features that a search strategy should exhibit to deal with dynamic optimization are diversity maintenance, a memory of past solutions, and a multipopulation structure of candidate solutions. In this paper, an immune-inspired algorithm that presents these features, called dopt-aiNet, is properly adapted to deal with six newly proposed benchmark instances, and the obtained results are outlined according to the available specifications for the competition at the Congress on Evolutionary Computation 2009.

This paper proposes a new variant of the PSO algorithm named complex neighborhood particle swarm optimizer (CNPSO) for solving global optimization problems. In the CNPSO, the neighborhood of the particles is organized through a complex network which is modified during the search process. This evolution of the topology seeks to improve the influence of the most successful particles and it is fine tuned for maintaining the scale-free characteristics of the network while the optimization is being performed. The use of a scale-free topology instead of the usual regular or global neighborhoods is intended to bring to the search procedure a better capability of exploring promising regions without a premature convergence, which would result in the procedure being easily trapped in a local optimum. The performance of the CNPSO is compared with the standard PSO on some well-known and high-dimensional benchmark functions, ranging from multimodal to plateau-like problems. In all the cases the CNPSO outperformed the standard PSO.

Bayesian networks have been widely applied to the feature selection problem. The existing approaches learn a Bayesian network from the available dataset and, afterward, utilize the Markov Blanket of the target feature as the criterion to select the relevant features. The Bayesian network learning can be viewed as a search and optimization procedure, where a search mechanism explores the space of all network structures while a scoring metric evaluates each candidate solution based on the likelihood. This paper investigates the application of an immune-inspired algorithm as the search procedure for obtaining high-quality Bayesian networks, motivated by the dynamical control of the population size and diversity along the search. Due to the resulting multimodal search capability, in a single run of the algorithm several subsets of features are obtained. Experiments on ten datasets were carried out in order to evaluate the proposed methodology in classification problems, and reduced-size subsets of features were produced.

This paper presents an artificial homeostatic system (AHS) devoted to the autonomous navigation of mobile robots, with emphasis on neuro-endocrine interactions. The AHS is composed of two modules, each one associated with a particular reactive task and both implemented using an extended version of the GasNet neural model, denoted spatially unconstrained GasNet model or simply non-spatial GasNet (NSGasNet). There is a coordination system, which is responsible for the specific role of each NSGasNet at a given operational condition. The switching among the NSGasNets is implemented as an artificial endocrine system (AES), which is based on a system of coupled nonlinear difference equations. The NSGasNets are synthesized by means of an evolutionary algorithm. The obtained neuro-endocrine controller is adopted in simulated and real benchmark applications, and the additional flexibility provided by the use of NSGasNet, together with the existence of an automatic coordination system, guides to convincing levels of performance.

This work introduces an ant-inspired algorithm for optimization in continuous search spaces that is based on the generation of random vectors with multivariate Gaussian pdf. The proposed approach is called MACACO -- Multivariate Ant Colony Algorithm for Continuous Optimization -- and is able to simultaneously adapt all the dimensions of the random distribution employed to generate the new individuals at each iteration. In order to analyze MACACO's search efficiency, the approach was compared to a pair of counterparts: the Continuous Ant Colony System (CACS) and the approach known as Ant Colony Optimization in en (ACOR). The comparative analysis, which involves well-known benchmark problems from the literature, has indicated that MACACO outperforms CACS and ACOR in most cases as the quality of the final solution is concerned, and it is just about two times more costly than the least expensive contender.

This paper proposes the application of a novel bio-inspired algorithm as a search engine to the feature subset selection problem. We may interpret our algorithm as an estimation of distribution algorithm that adopts an artificial immune system to implement the search process in the space of all features and a Bayesian network to implement the probabilistic model of the promising solutions. The characteristics of the proposed algorithm are the capability of effectively identifying and manipulating building blocks, maintenance of diversity in the population, and automatic control of the population size. These properties allow the algorithm to perform a multimodal search, known to be of great relevance in feature selection problems. Experiments on five datasets were carried out in order to evaluate the proposed methodology in classification problems and its performance compares favorably to that produced by contenders.

Despite thousands of years of changes in medical practice, healthcare delivery remains highly dependent on manual human effort. Mobile robots can help clinicians by automating the execution of tasks that do not directly demand medical knowledge (e.g. transporting medications to nurses). Yet, healthcare is a dynamic environment with a constantly mobile workforce. The present work describes a solution to the problem of autonomous robotic tracking of mobile targets in large, dynamic environments supported by a high-resolution, real-time, ultra wideband radio-frequency localization technology. The solution consists of a navigation system able to perform both global and local path planning simultaneously based on an evolutionary computation approach. A priori information and instant sensorial stimuli are integrated by the system in order to evolve efficient trajectories in real-time. The system proposed was tested with dynamic obstacles and targets and was demonstrated to be both highly adaptive and responsive.

Collaborative filtering (CF) is a method to perform automated suggestions for a user based on the opinion of other users with similar interest. Most of the CF algorithms do not take into account the existent duality between users and items, considering only the similarities between users or only the similarities between items. The authors have proposed in a previous work a bio-inspired methodology for CF, namely BIC-aiNet, capable of clustering rows and columns of a data matrix simultaneously. The usefulness and performance of the methodology are reported in the literature. Now, the authors carry out more rigorous comparative experiments with BIC-aiNet and other techniques found in the literature, as well as evaluate the scalability of the algorithm in several datasets of different sizes. The results indicate that our proposal is able to provide useful recommendations for the users, outperforming other methodologies for CF.

Constructive algorithms have shown to be reliable and effective methods for designing artificial neural networks (ANN) with good accuracy and generalization capability, yet with parsimonious network structures. Projection pursuit learning (PPL) has demonstrated great flexibility and effectiveness in performing this task, though presenting some difficulties in the search for appropriate projection directions in input spaces with high dimensionality. Due to the existence of high-dimensional input spaces in the context of time series prediction, mainly under the existence of long-term dependencies in the time series, we propose here a method based on the wrapper methodology to perform variable selection, so that only a subset of highly-informative lags is going to be considered as the regression vector. The yearly sunspot number time series is adopted as a case study and comparative analysis is performed considering alternative approaches in the literature, guiding to competitive results.

In an attempt to implement long-term time series prediction based on the recursive application of a one-step-ahead multilayer neural network predictor, we have considered the eleven short time series provided by the organizers of the Special Session NN3 Neural Network Forecasting Competition, and have proposed a joint application of a variable selection technique and a clustering procedure. The purpose was to define unbiased partition subsets and predictors with high generalization capability, based on a wrapper methodology. The proposed approach overcomes the performance of the predictor that considers all the lags in the regression vector. After obtaining the eleven long-term predictors, we conclude the paper presenting the eighteen multi-step predictions for each time series, as requested in the competition.

Collaborative filtering (CF) is a method to perform automated suggestions for a user based on the opinion of other users with similar interest. Most of the CF algorithms do not take into account the existent duality between users and items, considering only the similarities between users or only the similarities between items. In this paper we propose a novel methodology for the CF capable of dealing with this situation. By proposing an immune-inspired bi clustering technique to carry out clustering of rows and columns at the same time, our algorithm is able to group similarities between users and items. In order to evaluate the proposed methodology, we have applied it to Movie Lens dataset which contains user's ratings to a large set of movies. The results indicate that our proposal is able to provide useful recommendations for the users, outperforming other methodologies for CF reported in the literature.

This work presents the application of the omni-aiNet al- gorithm - an immune-inspired algorithm originally devel- oped to solve single and multiobjective optimization prob- lems - to the reconstruction of phylogenetic trees. The main goal of this work is to automatically evolve a population of phylogenetic unrooted trees, possibly with distinct topolo- gies, by minimizing at the same time the minimal evolution and the mean-squared error criteria. The obtained set of phylogenetic trees contains non-dominated individuals that form the Pareto front and that represent the trade-off of the two conflicting objectives. Given this set of phylogenetic trees, two multicriterion decision-making techniques were applied in order to try to select the best solution within the Pareto front.

Support vector machines (SVMs) have established themselves as a state-of-the-art technique for coping with non-trivial machine learning problems. Among the SVM variants, least-squares SVMs have gained increased attention recently due to the computational benefits they usually entail. Although considered as high-performance models, it is consensual that the applicability of these vector machines depends very much on a proper choice of some control parameters. In this paper, we present a sensitivity analysis study contrasting the performance profiles exhibited by standard and least-squares SVM classifiers with respect to the calibration of the kernel parameter value alone. The results achieved with simulations involving seven datasets indicate that the performance profiles are usually qualitatively similar for the two types of vector machines, both presenting kernel parameter values clearly associated with a better performance, and that the choice of the kernel function seems to be more critical than that of its parameter value.

The design of an autonomous navigation system with multiple tasks to be accomplished in unknown environments represents a complex undertaking. With the simultaneous purposes of capturing targets and avoiding obstacles, the challenge may become still more intricate if the configuration of obstacles and targets creates local minima, like concave shapes and mazes between the robot and the target. Pure reactive navigation systems are not able to deal properly with such hampering scenarios, requiring additional cognitive apparatus. Concepts from immune network theory are then employed to convert an earlier reactive robot controller, based on learning classifier systems, into a connectionist device. Starting from no a priori knowledge, both the classifiers and their connections are evolved during the robot navigation. Some experiments with and without local minima are carried out and the proposed evolutionary network of classifiers was shown to produce connectionist navigation systems capable of successfully overcoming local minima.

Autonomous robot navigation involves many challenges and difficulties which are augmented when multiple robots operate together. Sophisticated computational techniques are required to cope with autonomous navigation in collective robotics, being the biologically-inspired approaches the most frequently adopted. Stigmergy, i.e. the ants communication by means of pheromones, is the main biological metaphor used in this work to perform multi-robot communication. The robots will be able to mark regions of the environment with artificial pheromones, according to past experiences, assisting one another in a cooperative and indirect way to accomplish the navigation objectives. Each robot is controlled by an autonomous navigation system (ANS) based on learning classifier system, which evolves during navigation from no a priori knowledge. Besides learning to avoid obstacles and capture targets, the systems must also learn how and where to lay artificial pheromones. Some experiments and simulations are performed intending to particularly investigate the ANS from three main perspectives: capability of learning to achieve the navigation objectives in collective scenarios, adaptability in face of environmental changes and ability to obtain optimized navigation behaviors by means of stigmergy.

Multimodal optimization algorithms inspired by the immune system are generally characterized by a dynamic control of the population size and by diversity maintenance along the search. One of these proposals, denoted copt-aiNet (artificial immune network for combinatorial optimization), is used to deal with combinatorial problems like the Traveling Salesman Problem (TSP) and other permutation problems. In this paper, the copt-aiNet algorithm is extended and adapted to be applied to an important issue of modern data mining, the biclustering problem. The biclustering approach consists in simultaneously ordering the rows and columns of a given matrix, so that similar elements are grouped together. To illustrate the performance of the proposed method, two bitmap images are scrambled and used as input to the algorithm, and the biclustering procedure tries to restore the original image by grouping the pixels according to the similarity of colors in a neighborhood. Additionally, copt-aiNet is applied to gene expression data clustering, a classical problem of the bioinformatics literature, and its performance is compared with a hierarchical biclustering algorithm.

Multimodal optimization algorithms are being adapted to deal with dynamic optimization, mainly due to their ability to provide a faster reaction to unexpected changes in the optimization surface. The faster reaction may be associated with the existence of two important attributes in population-based algorithms devoted to multimodal optimization: simultaneous maintenance of multiple local optima in the population; and self-regulation of the population size along the search. The optimization surface may be subject to variations motivated by one of two main reasons: modification of the objectives to be fulfilled and change in parameters of the problem. An immune-inspired algorithm specially designed to deal with combinatorial optimization is applied here to solve time-varying TSP instances, with the cost of going from one city to the other being a function of time. The proposal presents favorable results when compared to the results produced by a high-performance ant colony optimization algorithm of the literature.

In this work, we propose an evolutionary-like approach to the problem of blind adaptive spatial filtering that is based on the decision-directed criterion and on the dopt-aiNet, an artificial immune network conceived to perform multimodal search in dynamic environments. The proposal was tested under static and time-varying undermodeled channel models, and, in all cases, its ability to find and track a solution close to the Wiener global optimum was attested. The obtained results reveal that the dopt-aiNet may decisively enhance the performance of adaptive arrays in scenarios built from elements that are representative of some aspects of real-world communication systems.

Radial basis function (RBF) neural networks are universal approximators and have been used for a wide range of applications. Aiming at reducing the number of neurons in the hidden layer, for regularization purposes, the center and dispersion of each RBF have to be properly defined by means of competitive learning. Only the output weights will be defined in a supervised manner. One of the drawbacks of such learning methodology, involving unsupervised and supervised learning, is that the centers will be defined so that regions in the input space with a high density of samples tend to be under-represented and those regions with a low density of samples tend to be over-represented. Additionally, few approaches provide a proper and individual indication of the dispersion of each RBF. This paper presents an immune density-preserving algorithm with adaptive radius, called ARIA, to determine the number of centers, their location and the dispersion of each RBF, based only on the available training data set. Considering classification problems, the algorithm to determine the hidden layer is compared to another immune-inspired algorithm called aiNet, K-means and the random choice of centers. The classification accuracy of the final network is compared to another density based approach and a decision tree classifier, C 5.0. The results are reported and analyzed.

Once the design of artificial neural networks (ANN) may require the optimization of numerical and structural parameters, bio-inspired algorithms have been successfully applied to accomplish this task, since they are population-based search strategies capable of dealing successfully with complex and large search spaces, avoiding local minima. In this paper, we propose the use of an artificial immune system for learning feedforward ANN's topologies. Besides the number of neurons in the hidden layer, the algorithm also optimizes the type of activation function for each node. The use of a Bayesian framework to infer the weights and weight decay terms as well as to perform model selection allows us to find neural models with high generalization capability and low complexity, once the Occam's razor principle is incorporated into the framework. We demonstrate the applicability of the proposal on seven classification problems and promising results were obtained.

In this paper, we propose the use of an immune-inspired approach called opt-aiNet to generate a diverse set of high-performance candidates to compose an ensemble of neural network classifiers. Being a population-based search algorithm, the opt-aiNet is capable of maintaining diversity and finding many high-performance solutions simultaneously, which are known to be desired features when synthesizing an ensemble. Concerning the selection and combination phases, the most relevant selection and combination techniques already proposed in the literature have been considered. The main contribution of this paper is the indication that there is no pair of selection/combination technique that can be considered the best one, because the performance of the obtained ensemble varies significantly with the current composition of the pool of candidates already produced by the generation phase. Notwithstanding, this variability in performance is not restricted to the choice of opt-aiNet as the generative device. As a consequence, to overcome the performance of the best individual classifier, every possible pairs of selection and combination techniques should be tried. Only with such an exhaustive search (notice that the main computational burden is usually related to the generation phase), the performance of the ensemble was invariably superior to the performance of the best individual classifier on four benchmark classification problems.

This paper proposes a supervised version of a learning algorithm for a constructive neuro-immune network. The proposed methodology is developed by taking ideas from the immune system and learning vector quantization. The resulting classification algorithm is characterized by high-performance, similar to the ones produced by alternative methods in the literature, and parsimonious solutions, with a much smaller set of prototypes per class when compared with the other approaches. The number of prototypes is automatically defined by the convergence criterion. The algorithm requires a single user-defined parameter for training, associated with the convergence criterion, and the computational cost is sufficiently reduced to support applications involving large data sets.

Support vector clustering (SVC) is a recently proposed clustering methodology with promising performance for high-dimensional and noisy datasets, and for clusters with arbitrary shape. This work addresses the application of SVC, a kernel-based method, in a context in which the channel equalization problem is conceived as a clustering task. The main challenge, in this case, is to perform unsupervised clustering aiming at the design of an optimal Bayesian or a blind prediction-based receiver without resorting to a priori information about the transmission medium. The proposed technique employs a two-stage procedure -a combination between the use of SVC to obtain a first set of clusters and an auxiliary heuristic to help separating eventual multiple clouds contained in a single cluster and attribute centers to them via an iterated local search (ILS) algorithm. The obtained results indicate that kernel methods can be successfully applied to the field of signal processing.

The purpose of this paper is a comparative study of a non-exhaustive, though representative, set of methodologies already available for the partition of the training dataset in time series prediction, and also for variable selection under the wrapper paradigm. The partition policy of the training dataset and the choice of a proper set of variables for the regression vector are known to have a significant influence in the accuracy of the predictor, no matter the choice of the prediction model. However, there has been no extensive search for a figure of merit supporting a comparative analysis. Here, two partition policies, denoted sequential and random, are compared, and among the variable selection approaches using wrappers, forward selection is contrasted with sensitivity based pruning. Five real financial time series with trends and seasonality have been considered and multilayer perceptrons are adopted as the predictor. The obtained results indicate with high confidence that the rarely adopted random partition and the computationally intensive forward selection overcomes the contestants in the whole set of experiments.

In this paper we present a methodology based on a combination of many distinct predictors in an ensemble, named hybrid ensemble model, to obtain a more accurate output using the results of single predictors. As basic components, we have used artificial neural networks and support vector machines models. In order to evaluate the performance, the hybrid model was required to predict a 24 h daily series energy consumption of a Brazilian electrical operation unit located in the northeast of Brazil. The proposed ensemble model has reached an error 25% smaller than that achieved by the best single predictor. The model was initialized several times to confirm that ensembles of predictors also tend to produce low variance profiles.

Research in collective robotics is motivated mainly by the possibility of achieving an efficient solution to multi-objective navigation tasks when multiple robots are employed, instead of a single robot. Several approaches have already been tried in multi-robot systems, but the bio-inspired ones are the most frequent. This paper proposes to augment an autonomous navigation system based on learning classifier systems for using in collective robotics, introducing an inter-robot communication mechanism inspired by ant stigmergy, with each robot acting independently and cooperatively. The navigation system has no innate basic behavior and all knowledge necessary to compose the decision-making artifact is evolved as a function of the environmental feedback only, during navigation. Repulsive and/or attractive pheromone trails are produced by the robots along navigation, following very simple rules. Basically, each robot has to perform obstacle avoidance and target search, and the status of the pheromone level at the position currently occupied by each robot will influence the coordination of the two fundamental behaviors. Experiments are performed in simulation, with comparative results indicating that the presence of the pheromone trails is responsible for significant improvements in the capture rate and in the length of the route adopted by each robot.

Multimodal optimization algorithms inspired by the immune system are generally characterized by a dynamic control of the population size and by diversity maintenance along the search. One of the most popular proposals is denoted opt-aiNet (artificial immune network for optimization) and is extended here to deal with time-varying fitness functions. Additional procedures are designed to improve the overall performance and the robustness of the immune-inspired approach, giving rise to a version for dynamic optimization, denoted dopt-aiNet. Firstly, challenging benchmark problems in static multimodal optimization are considered to validate the new proposal. No parameter adjustment is necessary to adapt the algorithm according to the peculiarities of each problem. In the sequence, dynamic environments are considered, and usual evaluation indices are adopted to assess the performance of dopt-aiNet and compare with alternative solution procedures available in the literature.

This paper proposes a novel constructive learning algorithm for a competitive neural network. The proposed algorithm is developed by taking ideas from the immune system and demonstrates robustness in the initial experiments reported here for a benchmark problem. Comparisons with results from the literature are also provided. To automatically segment the resultant neurons at the output, a tool from graph theory was used with promising results. General discussions and avenues for future works are also provided.

Bayesian networks learning from data has attracted a great deal of research. The usual approaches to accomplishing this task combine two elements. The first one is a heuristic search procedure to generate candidate solutions and the other element is a scoring metric to evaluate each obtained solution based on the likelihood of the network, that can be interpreted as a probability of observing the data set under a given network model. In this paper, we propose the use of an artificial immune system as the search procedure for obtaining high quality Bayesian networks, motivated by the multimodal search capability of these algorithms combined with the dynamical control of the population size and diversity along the search. We demonstrate the applicability of the proposal on two benchmarks and promising results were obtained.

Adaptive beamforming in antenna arrays aims at adjusting the weighted linear combination of the output signals provided by the antennas so that the power of the received signals at dominant paths is maximized at the same time that the power of interference and noise signals is minimized. The weight vectors, each one associated with one received signal can be directly obtained if the direction of arrival (DOA) of the corresponding signal has already been estimated. The process of DOA estimation involves the prediction of the angle of arrival by means of monitoring the output produced by the antennas in the array, given that the number of antennas is higher than the number of signals to be detected. Even though signal subspace techniques have made a good job in DOA estimation, they present some important drawbacks that are alleviated here using a supervised learning approach, in the form of a multiclass LS-SVM classification problem. The main contribution of this paper is twofold: a step-by-step description of the complete set of algebraic manipulation for data preprocessing and for the synthesis of the classification device, and an analysis of the effect in performance when relevant parameters vary in a given operational interval.

Prediction models for time series generally include preprocessing followed by the synthesis of an input-output mapping. Neural network models have been adopted to perform both steps, by means of unsupervised and supervised learning, respectively. The flexibility and the generalization capability are the most relevant attributes in favor of connectionist approaches. However, even though time series prediction can be roughly interpreted as learning from data, high levels of performance will solely be achieved if some peculiarities of each time series are properly considered in the design, particularly the existence of trend and seasonality. Instead of directly adopting detrend and/or deseasonality treatments, this paper proposes a novel paradigm for supervised learning based on a mixture of heterogeneous experts. Some mixture models have already been proved to produce good performance as predictors, but the present approach is devoted to a hybrid mixture composed of a set of distinct experts. The purpose is not only to further explore the "divide-and-conquer" principle, but also to compare the performance of mixture of heterogeneous experts with the standard mixture of experts approach, using ten distinct time series. The obtained results indicate that mixture of heterogeneous experts generally requires a more elaborate gating device and performs better in the case of more challenging time series.

This paper presents an autonomous evolutionary system applied to control a mobile robot in unknown environments. The navigation system learns efficiently to deal with situations where the robot must capture targets avoiding collisions with obstacles. Toward this end, robot direction and speed must be properly defined. The evolutionary approach is based on a version of classifier systems, responsible for the proposition of a competitive process involving rules of elementary behaviour. A virtual environment is used to evolve the controller, a Khepera II robot is submitted to real navigation tasks, with no significant degradation in performance. As an additional experiment, the controller is also evolved in a real environment, and validated in a different and more complex environment, not previously experimented, attesting the generalization capability of the proposal.

Several support vector machine (SVM) instances with distinct kernel functions may be separately created and properly combined into the same learning machine structure. This is the idea underlying heterogeneous ensembles of SVMs (HE-SVMs), an approach conceived to alleviate the performance bottlenecks incurred with the "kernel function choice" problem inherent in SVM design. In this paper, we assess the effectiveness of applying an evolutionary based mechanism (GASe1) in the search of the optimal subset of SVM models for automatic HE-SVM construction. GASe1 has the advantage of merging both the selection and combination of component SVMs into the same optimization process, and has shown sound performance when compared with two other component selection methods in complicated classification problems.

The design of autonomous navigation systems for mobile robots, with simultaneous objectives to be satisfied such as garbage collection with integrity maintenance, requires refined coordination mechanisms to deal with modules of elementary behaviour. This paper shows the implementation on a real Khepera II robot of an immuno-genetic network for autonomous navigation that combines an evolutionary algorithm with a continuous immune network model. The proposed immuno-genetic system has the immune network implementing a dynamic process of decision-making, and the evolutionary algorithm defining the network structure. To be able to evaluate the controllers (immune networks) on the evolutionary process, a virtual environment was used for computer simulation, based on the characteristics of the navigation problem. The immune networks obtained by evolution were then analyzed and tested on new situations, presenting coordination capability in simple and more complex tasks. Some preliminary experiments on a real Khepera II robot demonstrate the feasibility of the evolved immune networks.

In the present work, a constructive learning algorithm is employed to design an optimal one-hidden layer neural network structure that best approximates a given mapping. The method determines not only the optimal number of hidden neurons but also the best activation function for each node. Here, the projection pursuit technique is applied in association with the optimization of the solvability condition, giving rise to a more efficient and accurate computational learning algorithm. As each activation function of a hidden neuron is optimally defined for every approximation problem, better rates of convergence are achieved. The proposed constructive learning algorithm was successfully applied to identify a large-scale multivariate process, providing a multivariable model that was able to describe the complex process dynamics, even in long-range horizon predictions. The resulting identification model is then considered as part of a model-based predictive control strategy, with high-quality performance in closed-loop experiments.

The purpose of this paper is to introduce transductive inference with support vector machines (TSVM) as a powerful methodology for classification of gene expression data, using training and prediction data sets. The following classification problems will be considered: determination of cancer diagnosis categories and classification of genes from the budding yeast Saccharomyces cerevisiae in functional groups. In the case of training samples, experts have already classified the samples in their respective classes. So, given each prediction sample, the purpose is to determine its corresponding class. The main aspect of TSVM is that the classification task will be implemented in just one step, improving the generalization capability of the classifier. The TSVM will be compared with the traditional inductive method (SVM) in a series of experiments concerning the two classification problems, with promising results.

In this paper, we introduce a genetic algorithm-based training mechanism (HGT-GAME) toward the automatic structural design and parameter configuration of gated mixtures of experts (ME). In HGT-GAME, a whole ME instance is codified into a given chromosome. By employing regulatory genes, our approach enables the automatic pruning and growing of experts in a way to properly match the complexity of the task at hand. Moreover, to leverage HGT-GAME's effectiveness a local search refinement upon each ME chromosome is performed in each generation via the gradient descent-learning algorithm. Forecasting experiments evaluate the performance of gated MEs trained with HGT-GAME.

This work deals with the problem of scheduling jobs to identical parallel processors with the goal of minimizing the completion time of the last processor to finish its execution (makespan). This problem is known to be NP-Hard. The algorithm proposed here is inspired by the immune systems of vertebrate animals. The advantage of combinatorial optimization algorithms based on artificial immune systems is the inherent ability to preserve a diverse set of near-optimal solutions along the search. The results produced by the method are compared with results of classical heuristics.

This paper suggests a coevolutionary approach to design Takagi-Sugeno fuzzy models. The coevolutionary process induces cooperation among individuals of genetically different populations, the species. Populations from four species represent partial solutions to the fuzzy modeling problem. Cooperation is also achieved via fitness sharing, once the fitness of an individual depends on the fitness of individuals of different species. The performance of the proposed approach is evaluated using a function approximation problem with noisy data, and a classification problem.

This paper investigates an autonomous control system of a mobile robot based on the immune network theory. The immune network navigates the robot to solve a multiobjective task, namely, garbage collection: the robot must find and collect garbage, while it establishes a trajectory without colliding with obstacles, and return to the base before it runs out of energy. Each network node corresponds to a specific antibody and describes a particular control action for the robot. The antigens are the current state of the robot, read from a set of internal and external sensors. The network dynamics corresponds to the variation of antibody concentration levels, which change according to both mutual interaction of antibody nodes and of antibodies and antigens. It is proposed an evolutionary mechanism to determine the network configuration, that is, the parameters that define those interactions. Simulation results suggest that the proposal presented is very promising.

Evolutionary concepts are being used to support the implementation of higher level information processing devices than could be easily built by human design. We introduce an input device that proved to be appropriate for the study of human perception. The sequence of judgments can be captured to simulate an objective function. The repeated interaction between user and computer allows the user to search hyperspaces of possible solutions without being required to design equations by hand or even understand them. Two musical environments, Vox Populi, an evolutionary composition system, and InstrumentAll, a combined hardware and software musical interface, are briefly described.

Presents fuzzy modeling as a multi-objective decision making problem considering accuracy, interpretability and autonomy as goals. The proposed approach assumes that these goals can be handled via corresponding single-objective /spl epsiv/-constrained decision making problems whose solution is produced by a hierarchical evolutionary process. The fitting, generalization, and interpretation characteristics of the resulting fuzzy models are discussed using a classification problem.

This paper deals with a fuzzy-based system to solve the capacitated vehicle routing problem. The proposed method makes use of a neural network employing unsupervised learning guided by a fuzzy rule base. The algorithm is based on a policy of penalties and rewards, on a strategy of neuron inhibition, insertion and pruning, and on certain statistical characteristics of the input space. We make use of fuzzy theory aiming at minimizing drawbacks related to uncertainty and availability of partial information, and at synthesizing an adaptive process of constraint relaxation. The effectiveness of the proposed method is attested by means of a series of computational simulations comparing crisp and fuzzy approaches.

Neurofuzzy networks have become a powerful alternative strategy to develop fuzzy systems, since they are capable of learning and providing IF-THEN fuzzy rules in linguistic or explicit form. Amongst such models, ANFIS is recognized as a reference framework, mainly for its flexible and adaptive character. In this paper, we extend ANFIS theory by experimenting with a multi-net approach wherein two or more differently structured ANFIS instances are coupled to play together. Ensembles of ANFIS (E-ANFIS) enhance ANFIS performance skills and alleviate some of its computational bottlenecks. Moreover, they promote the automatic configuration of different ANFIS units and the a posteriori selective combination of their outputs. Experiments conducted to assess E-ANFIS generalization capability are also presented.

The clonal selection principle is used to explain the basic features of an adaptive immune response to an antigenic stimulus. It establishes the idea that only those cells that recognize the antigens (Ag's) are selected to proliferate. The selected cells are subject to an affinity maturation process, which improves their affinity to the selective Ag's. This paper proposes a computational implementation of the clonal selection principle that explicitly takes into account the affinity maturation of the immune response. The general algorithm, named CLONALG, is derived primarily to perform machine learning and pattern recognition tasks, and then it is adapted to solve optimization problems, emphasizing multimodal and combinatorial optimization. Two versions of the algorithm are derived, their computational cost per iteration is presented, and a sensitivity analysis in relation to the user-defined parameters is given. CLONALG is also contrasted with evolutionary algorithms. Several benchmark problems are considered to evaluate the performance of CLONALG and it is also compared to a niching method for multimodal function optimization.

This paper brings a detailed mathematical description of an artificial immune network model, named aiNet. The model is implemented in association with graph concepts and hierarchical clustering techniques, and is proposed to perform machine learning, data compression and cluster analysis. Pictorial representations for the aiNet basic units and typical architectures are introduced. The proposed immune network was primarily compared on a theoretical basis with well-known artificial neural networks. Then, the aiNet was applied to a non-linearly separable benchmark and a real-world problem, and the results were compared with that of the self-organizing feature map and with others already presented in the literature.

The appropriate operation of a radial basis function (RBF) neural network depends mainly upon an adequate choice of the parameters of its basis functions. The simplest approach to train an RBF network is to assume fixed radial basis functions defining the activation of the hidden units. Once the RBF parameters are fixed, the optimal set of output weights can be determined straightforwardly by using a linear least squares algorithm, which generally means reduction in the learning time as compared to the determination of all RBF network parameters using supervised learning. The main drawback of this strategy is the requirement of an efficient algorithm to determine the number, position, and dispersion of the RBFs. The approach proposed here is inspired by models derived from the vertebrate immune system, that will be shown to perform unsupervised cluster analysis. The algorithm is introduced and its performance is compared to that of the random, k-means center selection procedures and other results from the literature. By automatically defining the number of RBF centers, their positions and dispersions, the proposed method leads to parsimonious solutions. Simulation results are reported concerning regression and classification problems.

Improves some results associated with a modular and hierarchical evolutionary design of fuzzy systems, using a Takagi-Sugeno approach. Basically, the set of design parameters to be adjusted is divided into modules distributed over different levels that evolve in an interactive way. Due to the existence of a compromise between the flexibility of the fuzzy system architecture and the efficiency of the evolutionary process, a significant gain in performance can be obtained when the set of parameters to be automatically defined is divided into two groups: one optimized using a least square procedure, and the other evolved using genetic algorithms. Simulation results show that the proposed method increases computational tractability and favor the descriptive nature of the final solution.

We consider a neural network based fuzzy system model whose basic processing unit consists of two types of generic logic (OR and AND) neurons. The net is structured into a multilayer topology and trained by a competitive learning algorithm, together with a genetic algorithm approach to select the most suitable triangular norms and co-norms that model the logic neurons. The main features of the system include: automatic rule generation and selection, learning capability, processing time independent of the input space partition, and automatic selection of the t-norms and s-norms that model the basic logic operators (OR, AND) encountered in the theory of fuzzy sets. Four benchmark problems are considered to compare the performance of the proposed method with those produced by alternative strategies.

This paper presents an end-user interface that allows real time parametric control of sound events. It is an interactive environment in which evolutionary computation is applied to algorithmic composition. This system uses genetic algorithms to generate and evaluate a sequence of chords played as MIDI data. Melodic, harmonic and voice range fitness are used to control musical features. Based on the ordering of consonance of musical intervals, the notion of approximating a sequence of notes to its harmonically compatible note or tonal center is used. This method employs a fuzzy formalism and is posed as an optimization approach based on factors relevant to hearing music.

Tuning procedures for activation functions significantly increases the flexibility and the nonlinear approximation capability of feedforward neural networks in supervised learning tasks. As a consequence, the learning process presents a better performance, with the final state of the neural network being kept away from undesired saturation regions. Based on a hybrid architecture combining a gradient strategy with a fuzzy decision model, an auto-tuning algorithm is derived to adjust additional parameters associated with the activation functions. The other conventional parameters, the connection weights between layers, are adjusted using a powerful second-order approach based on a conjugate gradient algorithm. To demonstrate the performance of the proposed method we compare this technique with the standard algorithm and with an auto-tuning strategy based solely on the gradient descent method. The three algorithm are applied to several artificial and real world benchmarks.

Presents a pruning technique developed for the one-dimensional Kohonen self-organizing feature map (SOM) to be applied in clustering and classification problems. Its innovative aspect is the combined proposition of a penalty term, a clustering measure, a delayed pruning activation and a restarting phase. The proposed algorithm (PSOM) always guides to a reduced architecture capable of representing the data set. We compare the PSOM with the original SOM applying them to three different classification problems. The results show that the PSOM is able to present superior performance in all cases.

Considering computational algorithms available in the literature, associated with supervised learning in feedforward neural networks, a wide range of distinct approaches can be identified While the adjustment of the connection weights represents an omnipresent stage, the algorithms differ in three basic aspects: the technique chosen to determine the dimension of the multilayer neural network, the procedure adopted to determine the activation function of each neuron, and the kind of composition of the hidden activations used to produce the output. The advanced learning algorithms are designed to treat all these three aspects during learning, guiding to dedicated solutions. In this paper, an evolutionary hybrid learning algorithm is presented to deal simultaneously with these three aspects. The essence of this approach is the existence of a search procedure based on a synergy between genetic algorithms and conjugate gradient optimization.

Recurrent neural networks are dynamic nonlinear systems that can exhibit a wide range of behaviors. However, the availability of recurrent neural networks of practical importance is associated with the existence of efficient supervised learning algorithms based on optimization procedures for adjusting the parameters. To improve performance, second order information should be considered to minimize the error in the training process. The first objective of this work is to describe systematic ways of obtaining exact second-order information for a range of recurrent neural network configurations, with a low computational cost. The second objective is to present an improved version of the conjugate gradient algorithm that can be used to effectively explore the available second-order information.