Extracting important information from complex skin lesion images is vital to effectively distinguish between different types of skin cancer images. In addition to providing high classification performance, such computer-aided diagnostic methods are needed where the models are interpretable and can provide knowledge about the discriminative features in skin lesion images. This underlying information can significantly assist dermatologists in identifying a particular stage or type of cancer. With its flexible representation and global search abilities, Genetic Programming (GP) is an ideal learning al-gorithm to evolve interpretable models and identify important features with significant information to discriminate between skin cancer classes. This paper provides an in-depth analysis of a recent GP-based feature learning method where different well-developed feature descriptors are integrated into the learning algorithms to extract high-level features for skin cancer image classification. The study explores the effectiveness of utilizing feature learning for this complex task and designing program structure to suit the problem domain as it has shown promising results compared to commonly used feature descriptors and an existing GP-based feature learning method developed for general image classification. This study analyzes the GP-evolved models to identify the prominent features and most effective feature descriptors important for the classification of these skin cancer images. The evolved models are interpretable, they provide knowledge that can assist dermatologists in making diagnoses in real-time clinical situations by identifying prominent skin cancer characteristics captured by the feature descriptors and learned during the evolutionary process.

Early detection of cancer is vital for reducing mortality rates, but medical images come in various resolutions, often captured from diverse devices, and pose challenges due to high inter-class and intra-class variability. Integrating various feature descriptors enhances high-level feature extraction for improved classification. Having varied structure sizes of tumor characteristics in these medical images, extracting features from a single scale might not provide meaningful or discriminative features. Genetic Programming (GP) proves effective in this context due to its flexible representation and global search capabilities. Unlike existing GP methods relying on extracting features from a single scale of the input image, this paper introduces a novel GP-based feature learning approach that automatically selects scales and combines image descriptors for skin cancer detection. The method learns global features from diverse scales, leading to improved classification performance on dermoscopic and standard camera image datasets. The evolved solutions not only enhance classification but also pinpoint the most effective scales and feature descriptors for different skin cancer image datasets. The proposed method generates interpretable models, aiding medical practitioners in diagnoses by identifying cancer characteristics captured through automatically selected feature descriptors in the evolutionary process.

Malice scoring is a technique that is present throughout the literature to quantify a software malignance through the assignment of a malice score. However, the majority of existing malice scoring models are synthesised using manually selected features and weights, where a domain specialist is needed. Hence, this paper aim at utilising Genetic Programming and cooperative coevolution to automatically evolve an ensemble of symbolic regression functions to assign a malice score to an instance of software data. Using a publicly available dataset, the effectiveness of the proposed method is assessed and compared to that of the state-of-the-art malice scoring method. The experimental results show that the proposed method has significantly outperformed the benchmark method and exhibits the best-performing model that produces an overall balanced accuracy of 95.80%, correctly classifying 94.21% and 97.39% of unseen malicious and benign instances, respectively. Furthermore, various aspects of the proposed method and experimental results have been analysed in-depth to provide insight into the evolutionary process and some of the automatically evolved models.

Developing a computer-aided diagnostic system for detecting various skin malignancies from images has attracted many researchers. Unlike many machine-learning approaches, such as artificial neural networks, genetic programming (GP) automatically evolves models with flexible representation. GP successfully provides effective solutions using its intrinsic ability to select prominent features (i.e., feature selection) and build new features (i.e., feature construction). Existing approaches have utilized GP to construct new features from the complete set of original features and the set of operators. However, the complete set of features may contain redundant or irrelevant features that do not provide useful information for classification. This study aims to develop a two-stage GP method, where the first stage selects prominent features, and the second stage constructs new features from these selected features and operators, such as multiplication in a wrapper approach to improve the classification performance. To include local, global, texture, color, and multiscale image properties of skin images, GP selects and constructs features extracted from local binary patterns and pyramid-structured wavelet decomposition. The accuracy of this GP method is assessed using two real-world skin image datasets captured from the standard camera and specialized instruments, and compared with commonly used classification algorithms, three state of the art, and an existing embedded GP method. The results reveal that this new approach of feature selection and feature construction effectively helps improve the performance of the machine-learning classification algorithms. Unlike other black-box models, the evolved models by GP are interpretable; therefore, the proposed method can assist dermatologists to identify prominent features, which has been shown by further analysis on the evolved models.

Job Shop Scheduling (JSS) is an important real-world problem. However, the problem is challenging because of many conflicting objectives and the complexity of production flows. Genetic programming-based hyper-heuristic (GP-HH) is a useful approach for automatically evolving effective dispatching rules for many-objective JSS. However, the evolved Pareto-front is highly irregular, seriously affecting the effectiveness of GP-HH. Although the reference points method is one of the most prominent and efficient methods for diversity maintenance in many-objective problems, it usually uses a uniform distribution of reference points which is only appropriate for a regular Pareto-front. In fact, some reference points may never be linked to any Pareto-optimal solutions, rendering them useless. These useless reference points can significantly impact the performance of any reference-point-based many-objective optimization algorithms such as NSGA-III. This paper proposes a new reference point adaption process that explicitly constructs the distribution model using Gaussian process to effectively reduce the number of useless reference points to a low level, enabling a close match between reference points and the distribution of Pareto-optimal solutions. We incorporate this mechanism into NSGA-III to build a new algorithm called MARP-NSGA-III which is compared experimentally to several popular many-objective algorithms. Experiment results on a large collection of many-objective benchmark JSS instances clearly show that MARP-NSGA-III can significantly improve the performance by using our Gaussian Process-based reference point adaptation mechanism.

The performance of image classification is highly dependent on the quality of the extracted features that are used to build a model. Designing such features usually requires prior knowledge of the domain and is often undertaken by a domain expert who, if available, is very costly to employ. Automating the process of designing such features can largely reduce the cost and efforts associated with this task. Image descriptors, such as local binary patterns, have emerged in computer vision, and aim at detecting keypoints, for example, corners, line-segments, and shapes, in an image and extracting features from those keypoints. In this article, genetic programming (GP) is used to automatically evolve an image descriptor using only two instances per class by utilising a multitree program representation. The automatically evolved descriptor operates directly on the raw pixel values of an image and generates the corresponding feature vector. Seven well-known datasets were adapted to the few-shot setting and used to assess the performance of the proposed method and compared against six handcrafted and one evolutionary computation-based image descriptor as well as three convolutional neural network (CNN) based methods. The experimental results show that the new method has significantly outperformed the competitor image descriptors and CNN-based methods. Furthermore, different patterns have been identified from analysing the evolved programs.

Genetic programming (GP) is considered the most popular method for automatically discovering and constructing dispatching rules for scheduling problems. Pareto Local Search (PLS) is a simple and effective local search method for tackling multi-objective combinatorial optimization problems. Researchers have studied the application of PLS to multiobjective evolutionary algorithms (MOEAs) with some success. In fact, by hybridizing global search with local search, the performance of many MOEAs can be noticeably improved. Despite its preliminary success, the practical use of PLS in GP is relatively limited. In this study, our aim is to enhance the quality of evolved dispatching rules for many-objective Job Shop Scheduling (JSS) through hybridizing GP with PLS techniques and designing an effective selection mechanism of initial solutions for PLS. In this paper, we propose a new GP-PLS algorithm that investigates whether the fitness-based selection mechanism for selecting initial solutions for PLS can increase the chance of discovering highly effective dispatching rules for many-objective JSS. To evaluate the effectiveness of our new algorithm, GPPLS is compared with the current state-of-the-art algorithms for many-objective JSS. The experimental results confirm that the proposed method can outperform the four recently proposed algorithms because of the proper use of local search techniques.

Ensembles of classifiers have proved to be more effective than a single classification algorithm in skin image classification problems. Generally, the ensembles are created using the whole set of original features. However, some original features can be redundant and may not provide useful information in building good ensemble classifiers. To deal with this, existing feature construction methods that usually generate new features for only a single classifier have been developed but they fit the training data too well, resulting in poor test performance. This study develops a new classification method that combines feature construction and ensemble learning using genetic programming (GP) to address the above limitations. The proposed method is evaluated on two benchmark real-world skin image datasets. The experimental results reveal that the proposed algorithm has significantly outperformed two existing GP approaches, two state-of-the-art convolutional neural network methods, and ten commonly used machine learning algorithms. The evolved individual that is considered as a set of constructed features helps identify prominent original features which can assist dermatologists in making a diagnosis.

The exponential growth in the Internet of Things (IoT) market has led to the proliferation of cyber threats as millions of vulnerable IoT devices are connected to the Internet each year. Security practitioners and researchers capture attacks on IoT devices using honeypots to explore the attack process, identify the types of attacks and analyse the interaction of the attackers with IoT devices. Several studies have focused on the classification of attacks on IoT devices, however, they are limited to performing manual analysis on command data by assigning skill levels to the attackers and looking at the purpose of executing specific commands. In this paper, we report our analysis of the captured attacks on IoT devices for four months using a medium-interaction server honeypot. We extract a new feature set by analysing the attacks according to the depth of interaction by the attackers, their behaviour in the attack process and the resources they utilised to perform these attacks. We apply unsupervised learning (i.e. clustering) to automatically group captured attacks and build a model to highlight the important features that contribute to understanding the relationship between various attacks grouped in the same cluster.

The occurrence of malignant melanoma had enormously increased since past decades. For accurate detection and classification, not only discriminative features are required but a properly designed model to combine these features effectively is also needed. In this study, the multi-tree representation of genetic programming (GP) has been utilised to effectively combine different types of features and evolve a classification model for the task of melanoma detection. Local binary patterns have been used to extract pixel-level informative features. For incorporating the properties of ABCD (asymmetrical property, border shape, color variation and geometrical characteristics) rule of dermoscopy, various features have been used to include local and global information of the skin lesions. To meet the requirements of the proposed multi-tree GP representation, genetic operators such as crossover and mutation are designed accordingly. Moreover, a new weighted fitness function is designed to evolve better GP individuals having multiple trees influencing each other’s performance during the evolution, in order to get overall performance gains. The performance of the new method is checked on two benchmark skin image datasets, and compared with six widely used classification algorithms and the single tree GP method. The experimental results have shown that the proposed method has significantly outperformed all these classification methods.

Image classification is used for many tasks such as recognising handwritten digits, identifying the presence of pedestrians for self-driving cars, and even providing medical diagnosis from cell images. The current state-of-the-art solution for image classification, typically, uses convolutional neural networks (CNNs), however, there are limitations in this approach such as the need for manually crafted architectures and low interpretability. A genetic programming solution is proposed in this paper that aims to overcome these limitations, while also taking advantage of useful operators in CNNs such as convolutions and pooling. The new approach is tested on four widely used benchmark image datasets, and the experimental results show that the new method has achieved comparable performance to the state-of-the-art techniques. Furthermore, the automatically evolved programs are highly interpretable, and visualisations of those programs reveal interesting patterns.

Development of an effective skin cancer detection system can greatly assist the dermatologist while significantly increasing the survival rate of the patient. To deal with melanoma detection, knowledge of dermatology can be combined with computer vision techniques to evolve better solutions. Image classification can significantly help in diagnosing the disease by accurately identifying the morphological structures of skin lesions responsible for developing cancer. Genetic Programming (GP), an emerging Evolutionary Computation technique, has the potential to evolve better solutions for image classification problems compared to many existing methods. In this paper, GP has been utilized to automatically evolve a classifier for skin cancer detection and also analysed GP as a feature selection method. For combining knowledge of dermatology and computer vision techniques, GP has been given domain specific features provided by the dermatologists as well as Local Binary Pattern features extracted from the dermoscopic images. The results have shown that GP has significantly outperformed or achieved comparable performance compared to the existing methods for skin cancer detection.

Transfer learning is a machine learning technique which has demonstrated great success in improving outcomes on a broad range of problems. However prior methods of transfer learning in Genetic Programming (GP) have tended to rely on random processes or meta-knowledge of the problem structure to facilitate selection of information for use in transfer. To address these issues, a non-random method for automatically finding relevant information for transfer between two source domain problems from the same problem domain based on common subtrees is proposed. This information is then utilised within a modular transfer learning framework, being added to the function set for a target problem prior to population initialisation. The performance of the proposed method is assessed using multiple benchmark problems from two distinct problem domains, namely symbolic regression and Boolean domain problems, and compared to standard GP and the-state-of-the-art transfer learning method for the given problems. The results show that the newly introduced method has either significantly outperformed, or achieved comparable performance to, the competitor methods on the problems of the two domains. We conclude that the proposed method demonstrates ability as a general transfer learning technique for GP and note some possible avenues for future research based off these results.

In computer vision, training a model that performs classification effectively is highly dependent on the extracted features, and the number of training instances. Conventionally, feature detection and extraction are performed by a domain expert who, in many cases, is expensive to employ and hard to find. Therefore, image descriptors have emerged to automate these tasks. However, designing an image descriptor still requires domain-expert intervention. Moreover, the majority of machine learning algorithms require a large number of training examples to perform well. However, labeled data is not always available or easy to acquire, and dealing with a large dataset can dramatically slow down the training process. In this paper, we propose a novel genetic programming-based method that automatically synthesises a descriptor using only two training instances per class. The proposed method combines arithmetic operators to evolve a model that takes an image and generates a feature vector. The performance of the proposed method is assessed using six datasets for texture classification with different degrees of rotation and is compared with seven domain-expert designed descriptors. The results show that the proposed method is robust to rotation and has significantly outperformed, or achieved a comparable performance to, the baseline methods.

The goodness of the features extracted from the instances and the number of training instances are two key components in machine learning, and building an effective model is largely affected by these two factors. Acquiring a large number of training instances is very expensive in some situations such as in the medical domain. Designing a good feature set, on the other hand, is very hard and often requires domain expertise. In computer vision, image descriptors have emerged to automate feature detection and extraction; however, domain-expert intervention is typically needed to develop these descriptors. The aim of this paper is to utilize genetic programming to automatically construct a rotation-invariant image descriptor by synthesizing a set of formulas using simple arithmetic operators and first-order statistics, and determining the length of the feature vector simultaneously using only two instances per class. Using seven texture classification image datasets, the performance of the proposed method is evaluated and compared against eight domain-expert hand-crafted image descriptors. Quantitatively, the proposed method has significantly outperformed, or achieved comparable performance to, the competitor methods. Qualitatively, the analysis shows that the descriptors evolved by the proposed method can be interpreted.

Genetic programming (GP) is a well-known evolutionary computation technique, which has been successfully used to solve various problems, such as optimization, image analysis, and classification. Transfer learning is a type of machine learning approach that can be used to solve complex tasks. Transfer learning has been introduced to GP to solve complex Boolean and symbolic regression problems with some promise. However, the use of transfer learning with GP has not been investigated to address complex image classification tasks with noise and rotations, where GP cannot achieve satisfactory performance, but GP with transfer learning may improve the performance. In this paper, we propose a novel approach based on transfer learning and GP to solve complex image classification problems by extracting and reusing blocks of knowledge/information, which are automatically discovered from similar as well as different image classification tasks during the evolutionary process. The proposed approach is evaluated on three texture data sets and three office data sets of image classification benchmarks, and achieves better classification performance than the state-of-the-art image classification algorithm. Further analysis on the evolved solutions/trees shows that the proposed approach with transfer learning can successfully discover and reuse knowledge/information extracted from similar or different problems to improve its performance on complex image classification problems.

In the computer vision and pattern recognition fields, image classification represents an important yet difficult task. It is a challenge to build effective computer models to replicate the remarkable ability of the human visual system, which relies on only one or a few instances to learn a completely new class or an object of a class. Recently we proposed two genetic programming (GP) methods, one-shot GP and compound-GP, that aim to evolve a program for the task of binary classification in images. The two methods are designed to use only one or a few instances per class to evolve the model. In this study, we investigate these two methods in terms of performance, robustness, and complexity of the evolved programs. We use ten data sets that vary in difficulty to evaluate these two methods. We also compare them with two other GP and six non-GP methods. The results show that one-shot GP and compound-GP outperform or achieve results comparable to competitor methods. Moreover, the features extracted by these two methods improve the performance of other classifiers with handcrafted features and those extracted by a recently developed GP-based method in most cases.

Texture classification is an essential task in computer vision that aims at grouping instances that have a similar repetitive pattern into one group. Detecting texture primitives can be used to discriminate between materials of different types. The process of detecting prominent features from the texture instances represents a cornerstone step in texture classification. Moreover, building a good model using a few training instances is difficult. In this study, a genetic programming (GP) descriptor is proposed for the task of multiclass texture classification. The proposed method synthesises a set of mathematical formulas relying on the raw pixel values and a sliding window of a predetermined size. Furthermore, only two instances per class are used to automatically evolve a descriptor that has the potential to effectively discriminate between instances of different textures using a simple instance-based classifier to perform the classification task. The performance of the proposed approach is examined using two widely-used data sets, and compared with two GP-based and nine well-known non-GP methods. Furthermore, three hand-crafted domain-expert designed feature extraction methods have been used with the non-GP methods to examine the effectiveness of the proposed method. The results show that the proposed method has significantly outperformed all these other methods on both data sets, and the new method evolves a descriptor that is capable of achieving significantly better performance compared to hand-crafted features.

Genetic Programming (GP) has been applied to a wide range of image analysis tasks including many real-world segmentation problems. This paper introduces a new biological application of detecting Phormidium algae in rivers of New Zealand using raw images captured from the air. In this paper, we propose a GP method to the task of algae detection. The proposed method synthesises a set of image operators and adopts a simple thresholding approach to segmenting an image into algae and non-algae regions. Furthermore, the introduced method operates directly on raw pixel values with no human assistance required. The method is tested across seven different images from different rivers. The results show good success on detecting areas of algae much more efficiently than traditional manual techniques. Furthermore, the result achieved by the proposed method is comparable to the hand-crafted ground truth with a F-measure fitness value of 0.64 (where 0 is best, 1 is worst) on average on the test set. Issues such as illumination, reflection and waves are discussed.

Texture classification aims at categorising instances that have a similar repetitive pattern. In computer vision, texture classification represents a fundamental element in a wide variety of applications, which can be performed by detecting texture primitives of the different classes. Using image descriptors to detect prominent features has been widely adopted in computer vision. Building an effective descriptor becomes more challenging when there are only a few labelled instances. This paper proposes a new Genetic Programming (GP) representation for evolving an image descriptor that operates directly on the raw pixel values and uses only two instances per class. The new method synthesises a set of mathematical formulas that are used to generate the feature vector, and the classification is then performed using a simple instance-based classifier. Determining the length of the feature vector is automatically handled by the new method. Two GP and nine well-known non-GP methods are compared on two texture image data sets for texture classification in order to test the effectiveness of the proposed method. The proposed method is also compared to three hand-crafted descriptors namely domain-independent features, local binary patterns, and Haralick texture features. The results show that the proposed method has superior performance over the competitive methods.

In this paper, we propose a novel hybrid classification method which is based on two distinct approaches, namely Genetic Programming (GP) and Nearest Neighbour (kNN). The method relies on a memory list which contains some correctly labelled instances and is formed by classifiers evolved by GP. The class label of a new instance will be determined by combining its most similar instances in the memory list and the output of GP classifier on this instance. The results show that this proposed method can outperform conventional GP-based classification approach. Compared with conventional classification methods such as Naive Bayes, SVM, Decision Trees, and conventional kNN, this method can also achieve better or comparable accuracies on a set of binary problems. The evaluation cost of this hybrid method is much lower than that of conventional kNN.

Image classification is a complex but important task especially in the areas of machine vision and image analysis such as remote sensing and face recognition. One of the challenges in image classification is finding an optimal set of features for a particular task because the choice of features has direct impact on the classification performance. However the goodness of a feature is highly problem dependent and often domain knowledge is required. To address these issues we introduce a Genetic Programming (GP) based image classification method, Two-Tier GP, which directly operates on raw pixels rather than features. The first tier in a classifier is for automatically defining features based on raw image input, while the second tier makes decision. Compared to conventional feature based image classification methods, Two-Tier GP achieved better accuracies on a range of different tasks. Furthermore by using the features defined by the first tier of these Two-Tier GP classifiers, conventional classification methods obtained higher accuracies than classifying on manually designed features. Analysis on evolved Two-Tier image classifiers shows that there are genuine features captured in the programs and the mechanism of achieving high accuracy can be revealed. The Two-Tier GP method has clear advantages in image classification, such as high accuracy, good interpretability and the removal of explicit feature extraction process.