Violations of temporal memory safety ("use after free", "UAF") continue to pose a significant threat to software security. The CHERI capability architecture has shown promise as a technology for C and C++ language reference integrity and spatial memory safety. Building atop CHERI, prior works - CHERIvoke and Cornucopia - have explored adding heap temporal safety. The most pressing limitation of Cornucopia was its impractical "stop-the-world" pause times.

In computer vision, training a model that performs classification effectively is highly dependent on the extracted features, and the number of training instances. Conventionally, feature detection and extraction are performed by a domain expert who, in many cases, is expensive to employ and hard to find. Therefore, image descriptors have emerged to automate these tasks. However, designing an image descriptor still requires domain-expert intervention. Moreover, the majority of machine learning algorithms require a large number of training examples to perform well. However, labeled data is not always available or easy to acquire, and dealing with a large dataset can dramatically slow down the training process. In this paper, we propose a novel genetic programming-based method that automatically synthesises a descriptor using only two training instances per class. The proposed method combines arithmetic operators to evolve a model that takes an image and generates a feature vector. The performance of the proposed method is assessed using six datasets for texture classification with different degrees of rotation and is compared with seven domain-expert designed descriptors. The results show that the proposed method is robust to rotation and has significantly outperformed, or achieved a comparable performance to, the baseline methods.

The goodness of the features extracted from the instances and the number of training instances are two key components in machine learning, and building an effective model is largely affected by these two factors. Acquiring a large number of training instances is very expensive in some situations such as in the medical domain. Designing a good feature set, on the other hand, is very hard and often requires domain expertise. In computer vision, image descriptors have emerged to automate feature detection and extraction; however, domain-expert intervention is typically needed to develop these descriptors. The aim of this paper is to utilize genetic programming to automatically construct a rotation-invariant image descriptor by synthesizing a set of formulas using simple arithmetic operators and first-order statistics, and determining the length of the feature vector simultaneously using only two instances per class. Using seven texture classification image datasets, the performance of the proposed method is evaluated and compared against eight domain-expert hand-crafted image descriptors. Quantitatively, the proposed method has significantly outperformed, or achieved comparable performance to, the competitor methods. Qualitatively, the analysis shows that the descriptors evolved by the proposed method can be interpreted.

In the computer vision and pattern recognition fields, image classification represents an important yet difficult task. It is a challenge to build effective computer models to replicate the remarkable ability of the human visual system, which relies on only one or a few instances to learn a completely new class or an object of a class. Recently we proposed two genetic programming (GP) methods, one-shot GP and compound-GP, that aim to evolve a program for the task of binary classification in images. The two methods are designed to use only one or a few instances per class to evolve the model. In this study, we investigate these two methods in terms of performance, robustness, and complexity of the evolved programs. We use ten data sets that vary in difficulty to evaluate these two methods. We also compare them with two other GP and six non-GP methods. The results show that one-shot GP and compound-GP outperform or achieve results comparable to competitor methods. Moreover, the features extracted by these two methods improve the performance of other classifiers with handcrafted features and those extracted by a recently developed GP-based method in most cases.

Texture classification is an essential task in computer vision that aims at grouping instances that have a similar repetitive pattern into one group. Detecting texture primitives can be used to discriminate between materials of different types. The process of detecting prominent features from the texture instances represents a cornerstone step in texture classification. Moreover, building a good model using a few training instances is difficult. In this study, a genetic programming (GP) descriptor is proposed for the task of multiclass texture classification. The proposed method synthesises a set of mathematical formulas relying on the raw pixel values and a sliding window of a predetermined size. Furthermore, only two instances per class are used to automatically evolve a descriptor that has the potential to effectively discriminate between instances of different textures using a simple instance-based classifier to perform the classification task. The performance of the proposed approach is examined using two widely-used data sets, and compared with two GP-based and nine well-known non-GP methods. Furthermore, three hand-crafted domain-expert designed feature extraction methods have been used with the non-GP methods to examine the effectiveness of the proposed method. The results show that the proposed method has significantly outperformed all these other methods on both data sets, and the new method evolves a descriptor that is capable of achieving significantly better performance compared to hand-crafted features.

Texture classification aims at categorising instances that have a similar repetitive pattern. In computer vision, texture classification represents a fundamental element in a wide variety of applications, which can be performed by detecting texture primitives of the different classes. Using image descriptors to detect prominent features has been widely adopted in computer vision. Building an effective descriptor becomes more challenging when there are only a few labelled instances. This paper proposes a new Genetic Programming (GP) representation for evolving an image descriptor that operates directly on the raw pixel values and uses only two instances per class. The new method synthesises a set of mathematical formulas that are used to generate the feature vector, and the classification is then performed using a simple instance-based classifier. Determining the length of the feature vector is automatically handled by the new method. Two GP and nine well-known non-GP methods are compared on two texture image data sets for texture classification in order to test the effectiveness of the proposed method. The proposed method is also compared to three hand-crafted descriptors namely domain-independent features, local binary patterns, and Haralick texture features. The results show that the proposed method has superior performance over the competitive methods.

Dispatching rules have been commonly used in practice for making sequencing and scheduling decisions. Due to specific characteristics of each manufacturing system, there is no universal dispatching rule that can dominate in all situations. Therefore, it is important to design specialized dispatching rules to enhance the scheduling performance for each manufacturing environment. Evolutionary computation approaches such as tree-based genetic programming (TGP) and gene expression programming (GEP) have been proposed to facilitate the design task through automatic design of dispatching rules. However, these methods are still limited by their high computational cost and low exploitation ability. To overcome this problem, we develop a new approach to automatic programming via iterated local search (APRILS) for dynamic job shop scheduling. The key idea of APRILS is to perform multiple local searches started with programs modified from the best obtained programs so far. The experiments show that APRILS outperforms TGP and GEP in most simulation scenarios in terms of effectiveness and efficiency. The analysis also shows that programs generated by APRILS are more compact than those obtained by genetic programming. An investigation of the behavior of APRILS suggests that the good performance of APRILS comes from the balance between exploration and exploitation in its search mechanism.

In edge detection, a machine learning algorithm generally requires training images with their ground truth or designed outputs to train an edge detector. Meanwhile the computational cost is heavy for most supervised learning algorithms in the training stage when a large set of training images is used. To learn edge detectors without ground truth and reduce the computational cost, an unsupervised Genetic Programming (GP) system is proposed for low-level edge detection. A new fitness function is developed from the energy functions in active contours. The proposed GP system utilises single images to evolve GP edge detectors, and these evolved edge detectors are used to detect edges on a large set of test images. The results of the experiments show that the proposed unsupervised learning GP system can effectively evolve good edge detectors to quickly detect edges on different natural images.

Job Shop Scheduling (JSS) involves determining a schedule for processing jobs on machines to optimise some measure of delivery speed or customer satisfaction. We investigate a genetic programming based hyper-heuristic (GPHH) approach to evolving dispatching rules for a two-machine job shop in both static and dynamic environments. In the static case the proposed GPHH method can represent and discover optimal dispatching rules. In the dynamic case we investigate two representations (using a single rule at both machines and evolving a specialised rule for each machine) and the effect of changing the training problem instances throughout evolution. Results show that relative performance of these methods is dependent on the testing instances.

Order acceptance and scheduling (OAS) is a hard optimisation problem in which both acceptance decisions and scheduling decisions must be considered simultaneously. Designing effective solution methods or heuristics for OAS is not a trivial task, especially to deal with different problem configurations and sizes. This paper proposes a new heuristic framework called forward construction heuristic (FCH) for OAS and develops a new sequential genetic programming (SGPOAS) method for automatic design of FCHs. The key idea of the new GP method is to learn priority rules directly from optimal scheduling decisions at different decision moments and evolve a set of rules for FCHs instead of a single rule as shown in previous studies. The results show that evolved FCHs are significantly better than evolved single priority rules. The evolved FCHs are also competitive with the existing meta-heuristics in the literature and very effective for large problem instances.

Job Shop Scheduling (JSS) is a complex real-world problem aiming to optimise a measure of delivery speed or customer satisfaction by determining a schedule for processing jobs on machines. A major disadvantage of using a dispatching rule (DR) approach to solving JSS problems is their lack of a global perspective of the current and potential future state of the shop. We investigate a genetic programming based hyper-heuristic (GPHH) approach to develop "less-myopic" DRs for dynamic JSS. Results show that in the dynamic ten machine job shop, incorporating features of the state of the wider shop, and the stage of a job's journey through the shop, improves the mean performance, and decreases the standard deviation of performance of the best evolved rules.

Due-date assignment plays an important role in scheduling systems and strongly influences the delivery performance of job shops. Because of the stochastic and dynamic nature of job shops, the development of general due-date assignment models (DDAMs) is complicated. In this study, two genetic programming (GP) methods are proposed to evolve DDAMs for job shop environments. The experimental results show that the evolved DDAMs can make more accurate estimates than other existing dynamic DDAMs with promising reusability. In addition, the evolved operation-based DDAMs show better performance than the evolved DDAMs employing aggregate information of jobs and machines.

Edge detection is a subjective task. Traditionally, a moving window approach is used, but the window size in edge detection is a tradeoff between localization accuracy and noise rejection. An automatic technique for searching a discriminated pixel's neighbors to construct new edge detectors is appealing to satisfy different tasks. In this paper, we propose a genetic programming (GP) system to automatically search pixels (a discriminated pixel and its neighbors) to construct new low-level subjective edge detectors for detecting edges in natural images, and analyze the pixels selected by the GP edge detectors. Automatically searching pixels avoids the problem of blurring edges from a large window and noise influence from a small window. Linear and second-order filters are constructed from the pixels with high occurrences in these GP edge detectors. The experiment results show that the proposed GP system has good performance. A comparison between the filters with the pixels selected by GP and all pixels in a fixed window indicates that the set of pixels selected by GP is compact but sufficiently rich to construct good edge detectors.

Classification algorithms can suffer from performance degradation when the class distribution is unbalanced. This paper develops a two-step approach to evolving ensembles using genetic programming (GP) for unbalanced data. The first step uses multiobjective (MO) GP to evolve a Pareto-approximated front of GP classifiers to form the ensemble by trading-off the minority and the majority class against each other during learning. The MO component alleviates the reliance on sampling to artificially rebalance the data. The second step, which is the focus this paper, proposes a novel ensemble selection approach using GP to automatically find/choose the best individuals for the ensemble. This new GP approach combines multiple Pareto-approximated front members into a single composite genetic program solution to represent the (optimized) ensemble. This ensemble representation has two main advantages/novelties over traditional genetic algorithm (GA) approaches. First, by limiting the depth of the composite solution trees, we use selection pressure during evolution to find small highly-cooperative groups of individuals for the ensemble. This means that ensemble sizes are not fixed a priori (as in GA), but vary depending on the strength of the base learners. Second, we compare different function set operators in the composite solution trees to explore new ways to aggregate the member outputs and thus, control how the ensemble computes its output. We show that the proposed GP approach evolves smaller more diverse ensembles compared to an established ensemble selection algorithm, while still performing as well as, or better than the established approach. The evolved GP ensembles also perform well compared to other bagging and boosting approaches, particularly on tasks with high levels of class imbalance.

A scheduling policy strongly influences the performance of a manufacturing system. However, the design of an effective scheduling policy is complicated and time consuming due to the complexity of each scheduling decision, as well as the interactions among these decisions. This paper develops four new multi-objective genetic programming-based hyperheuristic (MO-GPHH) methods for automatic design of scheduling policies, including dispatching rules and due-date assignment rules in job shop environments. In addition to using three existing search strategies, nondominated sorting genetic algorithm II, strength Pareto evolutionary algorithm 2, and harmonic distance-based multi-objective evolutionary algorithm, to develop new MO-GPHH methods, a new approach called diversified multi-objective cooperative evolution (DMOCC) is also proposed. The novelty of these MO-GPHH methods is that they are able to handle multiple scheduling decisions simultaneously. The experimental results show that the evolved Pareto fronts represent effective scheduling policies that can dominate scheduling policies from combinations of existing dispatching rules with dynamic/regression-based due-date assignment rules. The evolved scheduling policies also show dominating performance on unseen simulation scenarios with different shop settings. In addition, the uniformity of the scheduling policies obtained from the proposed method of DMOCC is better than those evolved by other evolutionary approaches.

Basic features for edge detection, such as derivatives, can be further manipulated to improve detection performance. How to effectively combine different local features to improve detection performance remains an open issue and needs to be investigated. Genetic Programming (GP) has been employed to construct composite features. However, the range of the observations of an evolved program might be sparse and large, which is not good to indicate different edge responses. In this study, GP is used to construct composite features for edge detection via estimating the observations of evolved programs as triangular distributions. The results of the experiments show that the evolved programs with a large range of observations are not good to construct composite features. A proposed restriction on the range of the observations of evolved programs improves the performance of edge detection.

This paper focuses on order acceptance and scheduling (OAS) problem, where both acceptance and sequencing decisions have to be handled simultaneously. Because of its complexity, designing effective heuristics or meta-heuristics for OAS is challenging. This paper will investigate how genetic programming (GP) can be used to deal with OAS. The goal of this paper is to develop new GP frameworks to evolve high-performance scheduling rules/heuristics for OAS. The new frameworks are developed based on two key aspects: (1) separating acceptance and sequencing decisions, and (2) enhancing the quality of scheduling rules by embedding heuristic search mechanisms. The experimental results show that separating decisions is not trivial and can easily lead to overfitting issues. Meanwhile, embedding heuristic ideas into the scheduling rules can help search for better solutions for OAS.

Particle Swarm Optimization (PSO) is a population-based algorithm designed to find good solutions to optimization problems. Its characteristics have encouraged its adoption to tackle a variety of problems in different fields. However, when such problems are subject to noise, the performance of PSO suffers an immediate deterioration which demands the incorporation of noise handling mechanisms. One such mechanism comprises resampling methods, which re-evaluate the solutions multiple times in order to estimate their true objective values. The state-of-the-art integration with which the best results have been obtained utilizes the resampling method named Optimal Computing Budget Allocation (OCBA). This resampling method starts by estimating the objective values of all the solutions via Equal Resampling (ER), and then sequentially allocating further re-evaluations to the estimated best solutions. However, after having a first estimate via ER, we question the importance of the additional efforts to correctly select the true best solutions when a good-enough and accurate one can be selected. In this paper, we propose a new PSO algorithm based on ER in which the additional evaluations are allocated at once to the estimated best solutions, thus skipping the complexity of using OCBA. Experiments on 20 large-scale benchmark functions subject to different levels of noise show that the proposed algorithm produces similar results to PSO with OCBA in most cases.

The combination of local features in edge detection can generally improve detection performance. However, how to effectively combine different basic features remains an open issue and needs to be investigated. Multivariate density is a generalisation of the one-dimensional (univariate) distribution to higher dimensions. In order to effectively construct composite features with multivariate density, a Genetic Programming (GP) system is proposed to evolve Bayesian-based programs. An evolved Bayesian-based program estimates the relevant multivariate density to construct a composite feature. The results of the experiments show that the GP system constructs high-level combined features which substantially improve the detection performance.

Particle Swarm Optimization (PSO) is a population-based algorithm whose performance deteriorates in optimization problems subject to noise. An approach to mitigate the effect of noise is to incorporate resampling methods to evaluate the solutions multiple times and hence estimate better their objective values. The state of the art incorporates a resampling method named Optimal Computing Budget Allocation (OCBA) in order to improve the accuracy of the estimated best solutions. However, the state of the art spends over 95% of the function evaluations on OCBA while the remaining ones are left for PSO to find better solutions. In this paper, we investigate different distributions such that fewer function evaluations are spent on resampling and more on searching. Moreover, we develop a new algorithm in which the function evaluations spent on resampling are further utilized to provide a more robust updating mechanism in PSO via hypothesis testing. Experiments on large-scale function optimization problems with multiplicative Gaussian noise show that our approach has a better overall performance than the state of the art when resampling every two or more iterations. However, the state of the art finds the best solutions when resampling every iteration.

In classification, machine learning algorithms can suffer a performance bias when data sets are unbalanced. Data sets are unbalanced when at least one class is represented by only a small number of training examples (called the minority class), while the other class(es) make up the majority. In this scenario, classifiers can have good accuracy on the majority class, but very poor accuracy on the minority class(es). This paper proposes a multiobjective genetic programming (MOGP) approach to evolving accurate and diverse ensembles of genetic program classifiers with good performance on both the minority and majority of classes. The evolved ensembles comprise of nondominated solutions in the population where individual members vote on class membership. This paper evaluates the effectiveness of two popular Pareto-based fitness strategies in the MOGP algorithm (SPEA2 and NSGAII), and investigates techniques to encourage diversity between solutions in the evolved ensembles. Experimental results on six (binary) class imbalance problems show that the evolved ensembles outperform their individual members, as well as single-predictor methods such as canonical GP, naive Bayes, and support vector machines, on highly unbalanced tasks. This highlights the importance of developing an effective fitness evaluation strategy in the underlying MOGP algorithm to evolve good ensemble members.

Designing effective dispatching rules is an important factor for many manufacturing systems. However, this time-consuming process has been performed manually for a very long time. Recently, some machine learning approaches have been proposed to support this task. In this paper, we investigate the use of genetic programming for automatically discovering new dispatching rules for the single objective job shop scheduling problem (JSP). Different representations of the dispatching rules in the literature are newly proposed in this paper and are compared and analysed. Experimental results show that the representation that integrates system and machine attributes can improve the quality of the evolved rules. Analysis of the evolved rules also provides useful knowledge about how these rules can effectively solve JSP.

Edge detectors trained by a machine learning algorithm are usually evaluated by the accuracy based on overall pixels in the training stage, rather than the information for each training image. However, when the evaluation for training edge detectors considers the accuracy of each image, the influence on the final detectors has not been investigated. In this study, we employ genetic programming to evolve detectors with new fitness functions containing the accuracy of training images. The experimental results show that fitness functions based on the accuracy of single training images can balance the accuracies across detection results, and the fitness function combining the accuracy of overall pixels with the accuracy of training images together can improve the detection performance.

Genetic Programming (GP) has been used for edge detection, but there is no previous work that analyses the outputs from a GP detector before thresholding them to binary edge maps. When the threshold used in a GP system slightly changes, the final edge map from a detector may change a lot. Mapping the outputs of a GP detector to a grayscale space by a linear transformation is not effective. In order to address the problem of the sensitivity to the threshold values, we replace the linear transformation with an S-shaped transformation. We design two new fitness functions so that the outputs from an evolved detector can obtain better edge maps after mapping into a grayscale space. Experimental results show that the S-shaped transformation obtains soft edge maps similar to the fixed threshold and the new fitness functions improve the edge detection accuracy.

A scheduling policy (SP) strongly influences the performance of a manufacturing system. However, the design of an effective SP is complicated and time-consuming due to the complexity of each scheduling decision as well as the interactions between these decisions. This paper proposes novel multi-objective genetic programming based hyper-heuristic methods for automatic design of SPs including dispatching rules (DRs) and due-date assignment rules (DDARs) in job shop environments. The experimental results show that the evolved Pareto front contains effective SPs that can dominate various SPs from combinations of existing DRs with dynamic and regression-based DDARs. The evolved SPs also show promising performance on unseen simulation scenarios with different shop settings. On the other hand, the proposed Diversified Multi-Objective Cooperative Coevolution (DMOCC) method can effectively evolve Pareto fronts of SPs compared to NSGA-II and SPEA2 while the uniformity of SPs obtained by DMOCC is better than those evolved by NSGA-II and SPEA2.

Edge detection is an important area in computer vision and detecting continuous edges in noisy images is a hard problem. The Canonical Particle Swarm Optimisation (CanPSO) has been used for edge detection since 2009. Although the Bare Bones PSO (BBPSO) and the Fully Informed Particle Swarm (FIPS), as two well-known versions of PSO, have interesting features to overcome noise, they have never been applied to edge detection in noisy images. In this paper, six different static topologies along with two dynamic topologies are implemented within the three versions of PSO and their effects are investigated in a PSO-based edge detector in noisy images. Computational experiments show that FIPS with the toroidal topology outperforms the canonical and bare bones PSO with various static and dynamic topologies in most cases and is more robust to noise.

Single pixels can be directly used to construct low-level edge detectors but these detectors are not good for suppressing noise and some texture. In general, features based on a small area are used to suppress noise and texture. However, there is very little guidance in the literature on how to select the area size. In this paper, we employ Genetic Programming (GP) to evolve edge detectors via automatically searching for features based on flexible blocks rather than dividing a fixed window into small areas based on different directions. Experimental results for natural images show that using blocks to extract features obtains better performance than using single pixels only to construct detectors, and that GP can successfully choose the block size for extracting features.

Machine learning algorithms such as genetic programming (GP) can evolve biased classifiers when data sets are unbalanced. Data sets are unbalanced when at least one class is represented by only a small number of training examples (called the minority class) while other classes make up the majority. In this scenario, classifiers can have good accuracy on the majority class but very poor accuracy on the minority class(es) due to the influence that the larger majority class has on traditional training criteria in the fitness function. This paper aims to both highlight the limitations of the current GP approaches in this area and develop several new fitness functions for binary classification with unbalanced data. Using a range of real-world classification problems with class imbalance, we empirically show that these new fitness functions evolve classifiers with good performance on both the minority and majority classes. Our approaches use the original unbalanced training data in the GP learning process, without the need to artificially balance the training examples from the two classes (e.g., via sampling).

Edge detection is an important task in computer vision. This paper describes a global approach to edge detection using genetic programming (GP). Unlike most traditional edge detection methods which use local window filters, this approach directly uses an entire image as input and classifies pixels directly as edges or non-edges without preprocessing or postprocessing. Shifting operations and common standard operators are used to form the function set. Precision, recall and true negative rate are used to construct the fitness functions. This approach is examined and compared with the Laplacian and Sobel edge detectors on three sets of images providing edge detection problems of varying difficulty. The results suggest that the detectors evolved by GP outperform the Laplacian detector and compete with the Sobel detector in most cases.

Edge detection algorithms often produce broken edges, especially in noisy images. We propose an algorithm based on discrete particle swarm optimisation (PSO) to detect continuous edges in noisy images. A constrained PSO-based algorithm with a new objective function is proposed to address noise and reduce broken edges. The localisation accuracy of the new algorithm is compared with that of a modified version of the Canny algorithm as a Gaussian-based edge detector, the robust rank order (RRO)-based algorithm as a statistical based edge detector, and our previously developed PSO-based algorithm. Pratt's figure of merit is used as a measure of localisation accuracy for these edge detection algorithms. Experimental results show that the performance of the new algorithm is higher than the Canny and RRO algorithms in the images corrupted by two different types of noise (impulsive and Gaussian noise). The new algorithm also detects edges more accurately and smoothly than our previously developed algorithm in noisy images.

Machine learning algorithms can suffer a performance bias when data sets are unbalanced. This paper proposes a Multi-objective Genetic Programming approach using negative correlation learning to evolve accurate and diverse ensembles of non-dominated solutions where members vote on class membership. We also compare two popular Pareto-based fitness schemes on the classification tasks. We show that the evolved ensembles achieve high accuracy on both classes using six unbalanced binary data sets, and that this performance is usually better than many of its individual members.

Particle Swarm Optimisation (PSO) is an intelligent search method based on swarm intelligence and has been widely used in many fields. However it is also easily trapped in local optima. In order to find a global optimum, some evolutionary search operators used in multi-agent genetic algorithms are integrated into a novel hybrid PSO, with the expectation of effectively escaping from local optima. Particles share their history information and then update their positions using the latest and best history information. Some benchmark high-dimensional functions (from 20 to 10000 dimensions) are used to test the performance of the hybrid algorithms. The results demonstrate that the algorithm can solve high-dimensional nonlinear optimisation problems and that the number of function evaluations required to do so increases with function dimension at a sublinear rate.

Genetic programming based hyper-heuristics (GPHH) have become popular over the last few years. Most of these proposed GPHH methods have focused on heuristic generation. This study investigates a new application of genetic programming (GP) in the field of hyper-heuristics and proposes a method called GPAM, which employs GP to evolve adaptive mechanisms (AM) to solve hard optimisation problems. The advantage of this method over other heuristic selection methods is the ability of evolved adaptive mechanisms to contain complicated combinations of heuristics and utilise problem solving states for heuristic selection. The method is tested on three problem domains and the results show that GPAM is very competitive when compared with existing hyper-heuristics. An analysis is also provided to gain more understanding of the proposed method.

Detecting continuous edges is a hard problem especially in noisy images. We propose an algorithm based on particle swarm optimisation (PSO) to detect continuous and smooth edges in such images. A constrained PSO-based algorithm with a new penalised objective function and two constraints is proposed to overcome noise and reduce broken edges. The new algorithm is examined and compared with a modified version of the Canny algorithm, the robust rank order (RRO)-based algorithm, and an existing PSO-based algorithm on two sets of images with different types and levels of noise. The results suggest that the new algorithm detect edges more accurately than these three algorithms and the detected edges are smoother than those detected by the previous PSO algorithm and thinner than those detected by RRO.

In tree based Genetic Programming (GP) there is a tendency for program sizes to increase as the run proceeds without a corresponding improvement in fitness. This increases resource usage, both memory and CPU time, and may result in over-fitting the training data. Numerical simplification is a method for removing redundant code from the program trees as the run proceeds. Compared with the canonical genetic programming method, numerical simplification can generate much smaller programs, use much shorter evolutionary training times and achieve comparable effectiveness performance. A key parameter of this method is the simplification threshold. This paper examines whether there exists any relationship between the noise level in the input data and the optimum value for the simplification threshold and, if it exists, what that relationship is. Our results suggest that there is a relationship between the optimum value of the simplification threshold and the level of noise in the input data and that a lower bound for the optimum simplification threshold is equal to the noise level and an upper bound is five times the noise level.

In object detection, the goals of successfully discriminating between different kinds of objects (object classification) and accurately identifying the positions of all objects of interest in a large image (object localisation) are potentially in conflict. We propose a Multi-Objective Genetic Programming (MOGP) approach to the task of providing a decision-maker with a diverse set of alternative object detection programs that balance between high detection rate and low false-alarm rate. Experiments on two datasets, simple shapes and photographs of coins, show that it is difficult for a Single-Objective GP (SOGP) system (which weights the multiple objectives a priori) to evolve effective object detectors, but that an MOGP system is able to evolve a range of effective object detectors more efficiently.

Learning algorithms can suffer a performance bias when data sets are unbalanced. This paper proposes a Multi-Objective Genetic Programming (MOGP) approach using the accuracy of the minority and majority class as learning objectives. We focus our analysis on the classification ability of evolved Pareto-front solutions using the Area Under the ROC Curve (AUC) and investigate which regions of the objective trade-off surface favour high-scoring AUC solutions. We show that a diverse set of well-performing classifiers is simultaneously evolved along the Pareto-front using the MOGP approach compared to canonical GP where only one solution is found along the objective trade-off surface, and that in some problems the MOGP solutions had better AUC than solutions evolved with canonical GP using hand-crafted fitness functions.

This paper investigates improvements to the fitness function in Genetic Programming to better solve binary classification problems with unbalanced data. Data sets are unbalanced when there is a majority of examples for one particular class over the other class(es). We show that using overall classification accuracy as the fitness function evolves classifiers with a performance bias toward the majority class at the expense of minority class performance. We develop four new fitness functions which consider the accuracy of majority and minority class separately to address this learning bias. Results using these fitness functions show that good accuracy for both the minority and majority classes can be achieved from evolved classifiers while keeping overall performance high and balanced across the two classes.

This paper proposes a PSOAdaBoost algorithm incorporating particle swarm optimization within an AdaBoost framework for face detection applications. The basic component of an AdaBoost detector is a weak classifier, consisting of a feature, selected by an exhaustive search mechanism, and a decision threshold. The proposed PSOAdaBoost computes the best feature and optimizes the threshold in one optimization process. Experiments between the proposed algorithm and AdaBoost (with exhaustive feature selection) suggest that PSOAdaBoost has better performance in terms of much less training time and better classification accuracy.

This paper investigates the effect on building blocks during evolution of two online program simplification methods in genetic programming. The two simplification methods considered are algebraic simplification and numerical simplification. The building blocks considered are of a more general form (two and three level subtrees) than numeric constants only. Unlike most of the existing work which often uses simple symbolic regression tasks, this work considers classification tasks as examples. We develop a new method for encoding possible building blocks for the analysis. The results show that the two online program simplification methods can generate new diverse building blocks during evolution although they also destroy existing ones and that many of the existing building blocks are retained during evolution. Compared with the canonical genetic programming method, the two simplification methods can generate much smaller programs, use much shorter evolutionary training time and achieve comparable effectiveness performance.

This paper proposes and evaluates a Particle Swarm Optimization (PSO) based ensemble classifier. The members of the ensemble are Nearest Prototype Classifiers generated sequentially using PSO and combined by a majority voting mechanism. Two necessary requirements for good performance of an ensemble are accuracy and diversity of error. Accuracy is achieved by PSO minimizing a fitness function representing the error rate as the members are created. The diversity of error is promoted by using a different initialization of PSO each time to create a new member and by adopting decorrelated training where a penalty term is added to the fitness function to penalize particles that make the same errors as previously generated classifiers. Simulation experiments on different classification problems show that the ensemble has better performance than a single classifier and are effective in generating diverse ensemble members.

The standard tournament selection samples individuals with replacement. The sampling-with-replacement strategy has its advantages but also has issues. One of the commonly recognised issues is that it is possible to have some individuals not sampled at all during the selection phase. The not-sampled issue aggravates the loss of program diversity. However, it is not clear how the issue affects Genetic Programming (GP) search. This paper investigates the importance of the issue. The theoretical and experimental results show that the issue can be solved and the loss of diversity contributed by not-sampled individuals can be minimised. However, doing so does not appears to significantly improve a GP system. Our conclusion is that the not-sampled issue does not seriously affect the selection performance in the standard tournament selection.

Standard tournament selection samples individuals with replacement. The sampling-with-replacement strategy has its advantages but also has issues. One of the commonly recognised issues is that it is possible to have the same individual sampled multiple times in a tournament. Although the impact of this multi-sampled issue on genetic programming is not clear, some researchers believe that it may lower the probability of some good individuals being sampled or selected. One solution is to use an alternative tournament selection (no-replacement tournament selection), which samples individuals in a tournament without replacement. This paper analyses no-replacement tournament selection to investigate the impact of the scheme and the importance of the issue. Theoretical simulations show that when common tournament sizes and population sizes are used, no-replacement tournament selection does not make the selection behaviour significantly different from that in the standard one and that the multi-sampled issue seldom occurs. In general, the issue is not crucial to the selection behaviour of standard tournament selection.