Optimization is an important and decisive task in science. Many optimization problems in science are naturally too complicated and difficult to be modeled and solved by the conventional optimization methods such as mathematical programming problem solvers. Meta-heuristic algorithms that are inspired by nature have started a new era in computing theory to solve the optimization problems. The paper seeks to find an optimization algorithm that learns the expected quality of different places gradually and adapts its exploration-exploitation dilemma to the location of an individual. Using birds’ classical conditioning learning behavior, in this paper, a new particle swarm optimization algorithm has been introduced where particles can learn to perform a natural conditioning behavior towards an unconditioned stimulus. Particles are divided into multiple categories in the problem space and if any of them finds the diversity of its category to be low, it will try to go towards its best personal experience. But if the diversity among the particles of its category is high, it will try to be inclined to the global optimum of its category. We have also used the idea of birds’ sensitivity to the space in which they fly and we have tried to move the particles more quickly in improper spaces so that they would depart these spaces as fast as possible. On the contrary, we reduced the particles’ speed in valuable spaces in order to let them explore those places more. In the initial population, the algorithm has used the instinctive behavior of birds to provide a population based on the particles’ merits. The proposed method has been implemented in MATLAB and the results have been divided into several subpopulations or parts. The proposed method has been compared to the state-of-the-art methods. It has been shown that the proposed method is a consistent algorithm for solving the static optimization problems.

One of the most important assumptions in machine learning tasks is the fact that training data points and test data points are extracted from the same distribution. However, this paper assumes the situation in which this fact does no longer hold. Therefore, a task named space adjustment, through which the distribution of the data points in the training-data space and the distribution of the data points in the test-data space become identical, is inevitable. Hereby, authors propose a linear mapping for the space adjustment task in the paper. It considers four approaches for preserving localities among data samples during the space adjustment. Each approach is defined based on a different locality concept. Considering all locality concepts in an objective function, authors transform the space adjustment into an optimisation problem. The paper proposes to optimise the corresponding objective function by an iterative approach. Empirical study shows that the proposed method outperforms the baseline methods. To do experiments, authors employ a large number of real-world datasets.

This paper studies the cluster ensemble selection problem for unsupervised learning. Given a large ensemble of clustering solutions, our goal is to select a subset of solutions to form a smaller yet better performing cluster ensemble than using all available solutions. The common way of aggregating the chosen solutions is accumulating the information of the selected results to a similarity matrix. This paper suggests transforming the similarity matrix to a modularity matrix and then applying a new consensus function which optimizes modularity measure in it. We represent the modularity maximization problem as a 0-1 quadratic program which can be exactly solved for small datasets. We also established a new greedy algorithm, namely sum linkage, to optimize the objective function specially for large scale datasets in a very short time. We show that the proposed consensus partition gets much closer to the actual cluster structure than the partitions obtained from the direct application of common cluster ensemble methods. The promising results compared with other most cited consensus functions show the excellent efficiency of the proposed method.

In clustering ensemble, it is desired to combine several clustering outputs in order to create better results than the output results of the basic individual clustering methods in terms of consistency, robustness and performance. In this research, we want to present a clustering ensemble method with a new aggregation function. The proposed method is named Robust Clustering Ensemble based on Iterative Fusion of Base Clusters (RCEIFBC). This method takes into account the two similarity criteria: (a) one of them is the cluster-cluster similarity and (b) the other one is the object-cluster similarity. The proposed method has two steps and has been done on the binary cluster representation of the given ensemble. Indeed, before doing any step, the primary partitions are converted into a binary cluster representation where the primary ensemble has been broken into a number of primary binary clusters. The first step is to combine the primary binary clusters with the highest cluster-cluster similarity. This phase will be replicated as long as our desired candidate clusters are ready. The second step is to improve the merged clusters by assigning the data points to the merged clusters. The performance and robustness of the proposed method have been evaluated over different machine learning datasets. The experimentation indicates the effectiveness of the proposed method comparing to the state-of-the-art clustering methods in terms of performance and robustness.

Many stability measures, such as Normalized Mutual Information (NMI), have been proposed to validate a set of partitionings. It is highly possible that a set of partitionings may contain one (or more) high quality cluster(s) but is still adjudged a bad cluster by a stability measure, and as a resul t, is completely neglected. Inspired by evaluation approaches measuring the efficacy of a set of partitionings, researchers have tried to define new measures for evaluating a cluster. Thus far, the measures defined for assessing a cluster are mostly based on the well-known NMI measure. The drawback of this commonly used approach is discussed in this paper, after which a new asymmetric criterion, called the Alizadeh–Parvin–Moshki–Minaei criterion (APMM), is proposed to assess the association between a cluster and a set of partitionings. We show that the APMM criterion overcomes the deficiency in the conventional NMI measure. We also propose a clustering ensemble framework that incorporates the APMM's capabilities in order to find the best performing clusters. The framework uses Average APMM (AAPMM) as a fitness measure to select a number of clusters instead of using all of the results. Any cluster that satisfies a predefined threshold of the mentioned measure is selected to participate in an elite ensemble. To combine the chosen clusters, a co-association matrix-based consensus function (by which the set of resultant partitionings are obtained) is used. Because Evidence Accumulation Clustering (EAC) can not derive the co-association matrix from a subset of clusters appropriately, a new EAC-based method, called Extended EAC (EEAC), is employed to construct the co-association matrix from the chosen subset of clusters. Empirical studies show that our proposed approach outperforms other cluster ensemble approaches.

Conventional clustering ensemble algorithms employ a set of primary results; each result includes a set of clusters which are emerged from data. Given a large number of available clusters, one is faced with the following questions: (a) can we obtain the same quality of results with a smaller number of clusters instead of full ensemble? (b) If so, which subset of clusters is more efficient to be used in the ensemble? In this paper, these two questions are going to be answered. We explore a clustering ensemble approach combined with a cluster stability criterion as well as a dataset simplicity criterion to discover the finest subset of base clusters for each kind of datasets. Also, a novel method is proposed in order to accumulate the selected clusters and to extract final partitioning. Although it is expected that by reducing the size of ensemble the performance decreases, our experimental results show that our selecting mechanism generally lead to superior results.

In this paper, we present a novel optimization-based method for the combination of cluster ensembles. The information among the ensemble is formulated in 0-1 bit strings. The suggested model defines a constrained nonlinear objective function, called fuzzy string objective function (FSOF), which maximizes the agreement between the ensemble members and minimizes the disagreement simultaneously. Despite the crisp primary partitions, the suggested model employs fuzzy logic in the mentioned objective function. Each row in a candidate solution of the model includes membership degrees indicating how much data point belongs to each cluster. The defined nonlinear model can be solved by every nonlinear optimizer; however; we used genetic algorithm to solve it. Accordingly, three suitable crossover and mutation operators satisfying the constraints of the problem are devised. The proposed crossover operators exchange information between two clusters. They use a novel relabeling method to find corresponding clusters between two partitions. The algorithm is applied on multiple standard datasets. The obtained results show that the modified genetic algorithm operators are desirable in exploration and exploitation of the big search space.

Different classifiers with different characteristics and methodologies can complement each other and cover their internal weaknesses; so classifier ensemble is an important approach to handle the weakness of single classifier based systems. In this article we explore an automatic and fast function to approximate the accuracy of a given classifier on a typical dataset. Then employing the function, we can convert the ensemble learning to an optimisation problem. So, in this article, the target is to achieve a model to approximate the performance of a predetermined classifier over each arbitrary dataset. According to this model, an optimisation problem is designed and a genetic algorithm is employed as an optimiser to explore the best classifier set in each subspace. The proposed ensemble methodology is called classifier ensemble based on subspace learning (CEBSL). CEBSL is examined on some datasets and it shows considerable improvements.