Counterfactual explanation is a popular eXplainable AI technique, that gives contrastive explanations to answer potential "what-if" questions about the workings of machine learning models. However, research into how explanations are understood by human beings has shown that an optimal explanation should be both selected and social, providing multiple varying explanations for the same event that allow a user to select specific explanations based on prior beliefs and cognitive biases. In order to provide such explanations, a Rashomon set of explanations can be created: a set of explanations utilising different features in the data. Current work to generate counterfactual explanations does not take this need into account, only focusing on producing a single optimal counterfactual.

Explainable artificial intelligence (XAI) has received great interest in the recent decade, due to its importance in critical application domains, such as self-driving cars, law, and healthcare. Genetic programming (GP) is a powerful evolutionary algorithm for machine learning. Compared with other standard machine learning models such as neural networks, the models evolved by GP tend to be more interpretable due to their model structure with symbolic components. However, interpretability has not been explicitly considered in GP until recently, following the surge in the popularity of XAI. This article provides a comprehensive review of the studies on GP that can potentially improve the model interpretability, both explicitly and implicitly, as a byproduct. We group the existing studies related to explainable artificial intelligence by GP into two categories. The first category considers the intrinsic interpretability, aiming to directly evolve more interpretable (and effective) models by GP. The second category focuses on post-hoc interpretability, which uses GP to explain other black-box machine learning models, or explain the models evolved by GP by simpler models such as linear models. This comprehensive survey demonstrates the strong potential of GP for improving the interpretability of machine learning models and balancing the complex tradeoff between model accuracy and interpretability.

Counterfactual explanations are a popular eXplainable AI technique, used to provide contrastive answers to “what-if” questions. These explanations are consistent with the way that an everyday person will explain an event, and have been shown to satisfy the ‘right to explanation’ of the European data regulations. Despite this, current work to generate counterfactual explanations either makes assumptions about the model being explained or utlises algorithms that perform suboptimally on continuous data. This work presents two novel algorithms to generate counterfactual explanations using Particle Swarm Optimization (PSO) and Differential Evolution (DE). These are shown to provide effective post-hoc explanations that make no assumptions about the underlying model or data structure. In particular, PSO is shown to generate counterfactual explanations that utilise significantly fewer features to generate sparser explanations when compared to previous related work.

Manifold learning (MaL) methods are an invaluable tool in today’s world of increasingly huge datasets. MaL algorithms can discover a much lower-dimensional representation (embedding) of a high-dimensional dataset through nonlinear transformations that preserve the most important structure of the original data. State-of-the-art MaL methods directly optimize an embedding without mapping between the original space and the discovered embedded space. This makes interpretability—a key requirement in exploratory data analysis—nearly impossible. Recently, genetic programming has emerged as a very promising approach to MaL by evolving functional mappings from the original space to an embedding. However, genetic programming-based MaL has struggled to match the performance of other approaches. In this work, we propose a new approach to using genetic programming for MaL, which preserves local topology. This is expected to significantly improve performance on tasks where local neighborhood structure (topology) is paramount. We compare our proposed approach with various baseline MaL methods and find that it often outperforms other methods, including a clear improvement over previous genetic programming approaches. These results are particularly promising, given the potential interpretability and reusability of the evolved mappings.

Clustering is the process of grouping related instances of unlabelled data into distinct subsets called clusters. While there are many different clustering methods available, almost all of them use simple distance-based (dis)similarity functions such as Euclidean Distance. However, these and most other predefined dissimilarity functions can be rather inflexible by considering each feature equally and not properly capturing feature interactions in the data. Genetic Programming is an evolutionary computation approach that evolves programs in an iterative process that naturally lends itself to the evolution of functions. This paper introduces a novel framework to automatically evolve dissimilarity measures for a provided clustering dataset and algorithm. The results show that the evolved functions create clusters exhibiting high measures of cluster quality.

Manifold learning is a widely used technique for reducing the dimensionality of complex data to make it more understandable and more efficient to work with. However, current state-of-the-art manifold learning techniques - such as Uniform Manifold Approximation and Projection (UMAP) - have a critical limitation. They do not provide a functional mapping from the higher dimensional space to the lower-dimensional space, instead, they produce only the lower-dimensional embedding. This means they are "black-boxes" that cannot be used in domains where explainability is paramount. Recently, there has been work on using genetic programming to perform manifold learning with functional mappings (represented by tree/s), however, these methods are limited in their performance compared to UMAP. To address this, in this work we propose utilising UMAP to create functional mappings with genetic programming-based manifold learning. We compare two different approaches: one that uses the embedding produced by UMAP as the target for the functional mapping; and the other which directly optimises the UMAP cost function by using it as the fitness function. Experimental results reinforce the value of producing a functional mapping and show promising performance compared to UMAP. Additionally, we visualise two-dimensional embeddings produced by our technique compared to UMAP to further analyse the behaviour of each of the algorithms.

Data visualization is a key tool in data mining for understanding big datasets. Many visualization methods have been proposed, including the well-regarded state-of-the-art method t-distributed stochastic neighbor embedding. However, the most powerful visualization methods have a significant limitation: the manner in which they create their visualization from the original features of the dataset is completely opaque. Many domains require an understanding of the data in terms of the original features; there is hence a need for powerful visualization methods which use understandable models. In this article, we propose a genetic programming (GP) approach called GP-tSNE for evolving interpretable mappings from the dataset to high-quality visualizations. A multiobjective approach is designed that produces a variety of visualizations in a single run which gives different tradeoffs between visual quality and model complexity. Testing against baseline methods on a variety of datasets shows the clear potential of GP-tSNE to allow deeper insight into data than that provided by existing visualization methods. We further highlight the benefits of a multiobjective approach through an in-depth analysis of a candidate front, which shows how multiple models can be analyzed jointly to give increased insight into the dataset.

Clustering is a widely used unsupervised learning technique. However, as the size and complexity of data increases, the performance of clustering algorithms diminishes, as well as the interpretability of the clustering partition. Genetic programming has been used to perform feature construction on data to increase clustering performance. However, existing work has not focused on encouraging simpler constructed features. In this paper, existing techniques are further developed to include parsimony pressure-a method to encourage evolution towards simpler solutions. With simpler solutions, the constructed features become easier to understand and interpret. The results of experiments using the proposed method show that parsimony pressure is an effective method for producing significantly simpler constructed features without any reduction on the performance of k-means++clustering. Evolved individuals are also analysed to demonstrate the effect of parsimony pressure on interpretability, showing the power of parsimony pressure for avoiding redundancies in individuals, and thus increasing the interpretability.

Clustering is a difficult and widely studied data mining task, with many varieties of clustering algorithms proposed in the literature. Nearly all algorithms use a similarity measure such as a distance metric (e.g., Euclidean distance) to decide which instances to assign to the same cluster. These similarity measures are generally predefined and cannot be easily tailored to the properties of a particular dataset, which leads to limitations in the quality and the interpretability of the clusters produced. In this article, we propose a new approach to automatically evolving similarity functions for a given clustering algorithm by using genetic programming. We introduce a new genetic programming-based method which automatically selects a small subset of features (feature selection) and then combines them using a variety of functions (feature construction) to produce dynamic and flexible similarity functions that are specifically designed for a given dataset. We demonstrate how the evolved similarity functions can be used to perform clustering using a graph-based representation. The results of a variety of experiments across a range of large, high-dimensional datasets show that the proposed approach can achieve higher and more consistent performance than the benchmark methods. We further extend the proposed approach to automatically produce multiple complementary similarity functions by using a multi-tree approach, which gives further performance improvements. We also analyse the interpretability and structure of the automatically evolved similarity functions to provide insight into how and why they are superior to standard distance metrics.

Clustering, an important unsupervised learning task, is very challenging on high-dimensional data, since the generated clusters can be significantly less meaningful as the number of features increases. Feature selection and/or feature weighting can address this issue by selecting and weighting only informative features. These techniques have been extensively studied in supervised learning, e.g. classification, but they are very difficult to use with clustering due to the lack of effective similarity/distance and validation measures. This paper utilises the powerful global search ability of particle swarm optimisation (PSO) on continuous problems, to propose a PSO based method for simultaneous feature selection and feature weighting for clustering on high-dimensional data, where a new validation measure is also proposed as the fitness function of the PSO method. Experiments on datasets with varying dimensionalities and different number of known clusters show that the proposed method can successfully improve clustering performance of different types of clustering algorithms over using the baseline of the original feature set.

There has been a wealth of feature selection algorithms proposed in recent years, each of which claims superior performance in turn. A wide range of datasets have been used to compare these algorithms, each with different characteristics and quantities of redundant and noisy features. Hence, it is very difficult to comprehensively and fairly compare these feature selection methods in order to find which are most robust and effective. In this work, we examine using Genetic Programming to automatically synthesise redundant features for augmenting existing datasets in order to more scientifically test feature selection performance. We develop a method for producing complex multi-variate redundancies, and present a novel and intuitive approach to ensuring a range of redundancy relationships are automatically created. The application of these augmented datasets to well-established feature selection algorithms shows a number of interesting and useful results and suggests promising directions for future research in this area.

Genetic programming (GP) has been shown to be very effective for performing data mining tasks. Despite this, it has seen relatively little use in clustering. In this work, we introduce a new GP approach for performing graph-based (GPGC) non-hyper-spherical clustering where the number of clusters is not required to be set in advance. The proposed GPGC approach is compared with a number of well known methods on a large number of data sets with a wide variety of shapes and sizes. Our results show that GPGC is the most generalisable of the tested methods, achieving good performance across all datasets. GPGC significantly outperforms all existing methods on the hardest ellipsoidal datasets, without needing the user to pre-define the number of clusters. To our knowledge, this is the first work which proposes using GP for graph-based clustering.

Genetic Programming (GP) has been applied to a wide range of image analysis tasks including many real-world segmentation problems. This paper introduces a new biological application of detecting Phormidium algae in rivers of New Zealand using raw images captured from the air. In this paper, we propose a GP method to the task of algae detection. The proposed method synthesises a set of image operators and adopts a simple thresholding approach to segmenting an image into algae and non-algae regions. Furthermore, the introduced method operates directly on raw pixel values with no human assistance required. The method is tested across seven different images from different rivers. The results show good success on detecting areas of algae much more efficiently than traditional manual techniques. Furthermore, the result achieved by the proposed method is comparable to the hand-crafted ground truth with a F-measure fitness value of 0.64 (where 0 is best, 1 is worst) on average on the test set. Issues such as illumination, reflection and waves are discussed.