Zero-shot proxies, also known as training-free metrics, are widely adopted to reduce the computational overhead in neural network evaluation for scenarios such as Neural Architecture Search (NAS), as they do not require any training. Existing zero-shot metrics have several limitations, including weak correlation with the true performance and poor generalisation across different networks or downstream tasks. For example, most of these metrics apply only to either convolutional neural networks (CNNs) or Transformers, but not both. To address these limitations, we propose Sample-Wise Activation Patterns (SWAP), and its derivative, SWAP-Score, a novel and highly effective zero-shot metric. SWAP-Score is broadly applicable across both architecture families and task domains, demonstrating strong predictive performance in the majority of tasks. This metric measures the expressivity of neural networks over a mini-batch of samples, showing a high correlation with the neural networks' ground-truth performance. For both CNNs and Transformers, the SWAP-Score outperforms existing zero-shot metrics across computer vision and natural language processing tasks. For instance, Spearman's correlation coefficient between the SWAP-Score and CIFAR-10 validation accuracy for DARTS CNNs is 0.93, and 0.71 for FlexiBERT Transformers on GLUE tasks. Moreover, SWAP-Score is label-independent, hence can be applied at the pre-training stage of language models to estimate their performance for downstream tasks. When applied to NAS, SWAP-empowered NAS, SWAP-NAS can achieve competitive performance using only approximately 6 and 9 minutes of GPU time, on CIFAR-10 and ImageNet respectively.

Training-free metrics (a.k.a. zero-cost proxies) are widely used to avoid resource-intensive neural network training, especially in Neural Architecture Search (NAS). Recent studies show that existing training-free metrics have several limitations, such as limited correlation and poor generalisation across different search spaces and tasks. Hence, we propose Sample-Wise Activation Patterns and its derivative, SWAP-Score, a novel high-performance training-free metric. It measures the expressivity of networks over a batch of input samples. The SWAP-Score is strongly correlated with ground-truth performance across various search spaces and tasks, outperforming 15 existing training-free metrics on NAS-Bench-101/201/301 and TransNAS-Bench-101. The SWAP-Score can be further enhanced by regularisation, which leads to even higher correlations in cell-based search space and enables model size control during the search. For example, Spearman’s rank correlation coefficient between regularised SWAP-Score and CIFAR-100 validation accuracies on NAS-Bench-201 networks is 0.90, significantly higher than 0.80 from the second-best metric, NWOT. When integrated with an evolutionary algorithm for NAS, our SWAP-NAS achieves competitive performance on CIFAR-10 and ImageNet in approximately 6 minutes and 9 minutes of GPU time respectively.

Evolutionary neural architecture search (ENAS) has emerged as a promising approach to finding high-performance neural architectures. However, widespread application has been limited by the expensive computational costs due to the nature of evolutionary algorithms. In this study, we aim to significantly reduce the computational costs of ENAS by involving a training-free performance metric. Specifically, the network performance can be estimated by the training-free metric with only a single forward pass. However, training-free metrics have their own challenges, in particular, an insufficient correlation with ground-truth performance. We adopt a Graph Convolutional Network (GCN) based contrastive predictor which can leverage the low cost of the training-free performance metric yet improve the correlation between the estimated performance and the true performance of the candidate architectures. Combining a training-free metric - the number of linear regions with the GCN-based contrastive predictor and an active learning scheme, we propose Fast-ENAS which can achieve superior search efficiency and performance on the benchmark NAS-Bench-201 and DARTS search spaces. Furthermore, with a single GPU searching on the DARTS space, Fast-ENAS requires only 0.02 (29 minutes) and 0.026 (37 minutes) GPU days to achieve test error rates of 2.50% and 24.30% on CIFAR-10 and ImageNet respectively.

Neural architecture search (NAS) aims to automate architecture engineering in neural networks. This often requires a high computational overhead to evaluate a number of candidate networks from the set of all possible networks in the search space. Prediction of the performance of a network can alleviate this high computational overhead by mitigating the need for evaluating every candidate network. Developing such a predictor typically requires a large number of evaluated architectures which may be difficult to obtain. We address this challenge by proposing a novel evolutionary-based NAS strategy, predictor-assisted evolutionary NAS (PRE-NAS) which can perform well even with an extremely small number of evaluated architectures. PRE-NAS leverages new evolutionary search strategies and integrates high-fidelity weight inheritance over generations. Unlike one-shot strategies, which may suffer from bias in the evaluation due to weight sharing, offspring candidates in PRE-NAS are topologically homogeneous. This circumvents bias and leads to more accurate predictions. Extensive experiments on the NAS-Bench-201 and DARTS search spaces show that PRE-NAS can outperform state-of-the-art NAS methods. With only a single GPU searching for 0.6 days, a competitive architecture can be found by PRE-NAS which achieves 2.40% and 24% test error rates on CIFAR-10 and ImageNet, respectively.

Neural architecture search (NAS) aims to automate architecture engineering in neural networks. This often requires a high computational overhead to evaluate a number of candidate networks from the set of all possible networks in the search space during the search. Prediction of the networks' performance can alleviate this high computational overhead by mitigating the need for evaluating every candidate network. Developing such a predictor typically requires a large number of evaluated architectures which may be difficult to obtain. We address this challenge by proposing a novel evolutionary-based NAS strategy, Predictor-assisted E-NAS (PRE-NAS), which can perform well even with an extremely small number of evaluated architectures. PRE-NAS leverages new evolutionary search strategies and integrates high-fidelity weight inheritance over generations. Unlike one-shot strategies, which may suffer from bias in the evaluation due to weight sharing, offspring candidates in PRE-NAS are topologically homogeneous, which circumvents bias and leads to more accurate predictions. Extensive experiments on NAS-Bench-201 and DARTS search spaces show that PRE-NAS can outperform state-of-the-art NAS methods. With only a single GPU searching for 0.6 days, competitive architecture can be found by PRE-NAS which achieves 2.40% and 24% test error rates on CIFAR-10 and ImageNet respectively.

The purpose of convolutional neural networks is usually image classification but there are increasing studies attempting to reverse this common purpose in order to generate images. One of the most promising research directions is style transfer. This involves rendering the overall texture of an image into an artistic style. There are two common approaches in this field, which are feature representation based methods and generative adversarial network(GAN) based methods. In this paper, we focus on GAN based methods. We observed that most variants of GAN usually need paired data in order to generate the desired result, the training costs are very heavy and the quality of the result is not guaranteed. We propose an improved architecture for generative adversarial models for multi-style rendering. A new loss function configuration enables learning from unpaired data and generation of stylized images with specific artistic styles from normal photographs. A weighted combination of loss functions can control the trade-off between style and content of a stylized image.

Concept drift in data streams is a change in the underlying distribution that can cause algorithms that are classifying them to have increased error. There is a need for algorithms that can adapt to these changes. Genetic programming is one such algorithm that can adapt to streaming data however its use in this area is somewhat unexplored. It is hoped that because genetic programming is a population based method the variety of solutions tested every generation will enable it to adapt quickly. The adaptation speed can be increased by determining suitable parameters and settings. In order to test these ideas, experiments were run on several synthetic and one world streaming data set. The results found that genetic programming was capable of adapting quickly to concept drift and that increased rates of mutation and crossover can provide faster adaptation.

If output signals of artificial neural network classifiers are interpreted per node as class label predictors then partial knowledge encoded by the network during the learning procedure can be exploited in order to reassign which output node should represent each class label so that learning speed and final classification accuracy are improved. Our method for computing these reassignments is based on the maximum average correlation between actual node outputs and target labels over a small labeled validation dataset. Node Label Matching is an ancillary method for both supervised and unsupervised learning in artificial neural networks and we demonstrate its integration with Contrastive Divergence pre-training in Restricted Boltzmann Machines and Back Propagation fine-tuning in Deep Belief Networks. We introduce the Segmented Density Random Binary dataset and present empirical results of Node Label Matching on both our synthetic data and a subset of the MNIST benchmark.

We show a method that allows the pixel data of a set of images to be analyzed independently of any set of computed features. If the high and low aesthetic value images can be separated in the high dimensional space of pixel intensities then for any given set of features computed from the images, those features relevant to high aesthetic value can be determined and the range of feature values that correlate with high aesthetic appeal can be quantified. The method uses the Self Organizing Map to project raw pixel data of images onto a feature map. The aesthetic class of these images is overlayed on the feature map, yielding a semantic map. Average feature values are visualized in gray-scale heat maps and features relevant to aesthetic value are identified. We call this the Pixel Array Self Organizing Map (PASOM) method. For the set of images analyzed, brightness and texture features were identified as being discriminatory between images of high and low aesthetic value. High aesthetic value images tend to have higher brightness and richer textures. These findings were corroborated by a professional artist/photographer as being consistent with the principles for attaining aesthetic value in visual media. The PASOM method yields a semantic map and a visualization of feature value variation that together make possible a detailed analysis of features associated with the aesthetic value of images.

Activity recognition from smartphone sensor inputs is of great importance to enhance user experience. Our study aims to investigate the applicability of Genetic Programming (GP) approach on this complex real world problem. Traditional methods often require substantial human efforts to define good features. Moreover the optimal features for one type of activity may not be suitable for another. In comparison, our GP approach does not require such feature extraction process, hence, more suitable for complex activities where good features are difficult to be pre-defined. To facilitate this study we therefore propose a benchmark of activity data collected from various smartphone sensors, as currently there is no existing publicly available database for activity recognition. In this study, a GP-based approach is applied to nine types of activity recognition tasks by directly taking raw data instead of features. The effectiveness of this approach can be seen by the promising results. In addition our benchmark data provides a platform for other machine learning algorithms to evaluate their performance on activity recognition.

This paper describes an approach to the detection rice plants in images of rice fields by using genetic programming. The method involves the evolution of a genetic programming classifier of 20 × 20 pixel windows to distinguish rice and nonrice windows, applies the evolved classifier to each pixel position in a test image in a scanning window fashion and determines the class of a pixel by majority voting. The individual pixel values in the window comprise the terminal set. The four arithmetic operators, augmented by square root, comprise the function set. Fitness is a weighted sum of true positive and true negative rates. The classifier achieves an accuracy of 90% on positive and negative windows and is highly accurate in localizing rice leaves in test images for micro-spraying of nutritional supplements. The evolutionary approach clearly outperforms a thresholding approach based on colour which is unable to distinguish between rice an leaves.

Compared to conventional activity recognition methods using feature extraction followed by classification, the Genetic Programming (GP) based classification applied to raw sensor data can avoid the time-consuming and knowledge-dependent feature extraction procedure. However, the traditional GP-based classifier using accuracy as fitness function is sensitive to the choice of threshold values. Furthermore, sensor data of the same activity might demonstrate remarkable distinction when the signal is collected in the changing environment, which will lead to inconsistency between training and testing data and consequently degrade the generalization power of the trained classifier. Moreover, the GP-based classifier cannot well distinguish less separable activities in the presence of multiple activities. Our work aims to address these issues by improving the GP-based classifier via: (1) using the area under the receiver operating characteristic curve (AUC) as fitness function, (2) using an online local time series normalization procedure to pre-smooth undesirable features, and (3) using a binary tree based classification framework to force GP to learn key discriminating features that can better distinguish less separable activities. We test the proposed method on a sensor data set collected from a smartphone, consisting of four common human activities, sitting, standing, walking and running. The proposed GP-based classifier achieves the outstanding performance on recognizing each of four activities in terms of both high true positive and low false alarm rates, which much improves over the traditional GP-based classifier and several of its variants.

Recognition of activities such as sitting, standing, walking and running can significantly improve the interaction between human and machine, especially on mobile devices. In this study we present a GP based method which can automatically evolve recognition programs for various activities using multisensor data. This investigation shows that GP is capable of achieving good recognition on binary problems as well as on multi-class problems. With this method domain knowledge about an activity is not required. Furthermore, extraction of time series features is not necessary. The investigation also shows that these evolved GP solutions are small in size and fast in execution. They are suitable for real-world applications which may require real-time performance.

Our primary motivation in this paper is to determine whether evolved texture feature extraction programs are competitive with human derived programs for a difficult real world texture classification problem. The problem involves distinguishing images of three classes of bulk malt. There are subtle differences between the three classes. We have used a number of human derived methods, Haralick, Gabor, Haar, histogram and Galloway, to get feature vectors for the malt problem. We have also used a number of feature extraction programs that were evolved from thirteen Brodatz textures. We performed classification with a 1 nearest neighbour classifier. The evolved features gave an accuracy of 67% which is considerably better than the 53% achieved with the Haar features, but not as good as the 77% achieved with the Galloway features. Analysis of the evolved features suggested that they are capturing some texture regularities not captured by the human derived methods. We conclude that the evolved features are competitive with the human derived features and can provide enhanced accuracy when used in conjunction with human derived features.

The aim of event detection in time series is to identify particular occurrences of user-interest in one or more time lines, such as finding an anomaly in electrocardiograms or reporting a sudden variation of voltage in a power supply. Current methods are not adequate for detecting certain kinds of events without any domain knowledge. Therefore, we propose a Genetic Programming (GP) based event detection methodology in which solutions can be built from raw time series data. The framework is applied to five synthetic data sets and one real world application. The experimental results show that working on raw data even with a dimensionality as high as 140 × 80, genetic programming can achieve superior performance to conventional methods operating on pre-defined features. Furthermore, analysis of the evolved event detectors shows that they have captured the regularities inserted into the synthetic data sets and some individuals can be readily understood by humans.

The best evolutionary approach can be a difficult problem. In this work we have investigated two evolutionary representations to evolve non-photorealistic renderings: a variable-length classic genetic algorithm representation, and a tree-based genetic algorithm representation. These representations exhibit very different convergence behaviour, and despite considerable exploration of parameters the classic genetic algorithm was not competitive with the tree-based approach for the problem studied in this work. The aim of the work presented in this paper was to investigate whether analysis of the fitness landscapes described by the different representations can explain the difference in performance. We used several current fitness landscape measures to analyse the fitness landscapes, and found that one of the measures suggests there is a correlation between search performance and the fitness landscape.

At the current state of the art, genetic programs do not contain two constructs that commonly occur in programs written by humans, that is, loops and functions with parameters. In this paper we describe an investigation into the evolution of programs for a problem that can only be solved by evolving a parameterised program with one or more loops. We provide training examples of the desired program behaviour for a number of problem sizes and require the evolution of a program P(n) that will give the correct output for any value of n. We have chosen a problem, that of reproducing a binary string to a given number of bits, that can be made harder or easier by adjusting various aspects of the formulation. We are interested seeing in which formulations lead to success and which do not. We conclude that programs with parameters and loops can be successfully evolved if the search space is appropriately restricted by (1) grammars which restrict the possible program structures, (2) limits on program depth and (3) limits on the range of random constants.

We describe an approach to generating animations of drawings that start as a random collection of strokes and gradually resolve into a recognizable subject. The strokes are represented as tree based genetic programs. An animation is generated by rendering the best individual in a generation as a frame of a movie. The resulting animations have an engaging characteristic in which the target slowly emerges from a random set of strokes. We have generated two qualitatively different kinds of animations, ones that use grey level straight line strokes and ones that use binary Bezier curve stokes. Around 100,000 generations are needed to generate engaging animations. Population sizes of 2 and 4 give the best convergence behaviour. Convergence can be accelerated by using information from the target in drawing a stroke. Our approach provides a large range of creative opportunities for artists. Artists have control over choice of target and the various stroke parameters.

A traditional photomosaic is a still image where a larger picture is created by selectively arranging small picture tiles on a blank, gridded canvas. We show interesting and engaging animations can be generated from an evolutionary search for the final photomosaic image. We then investigate two different tile placement strategies for generating the animations. In the first strategy tiles can only be placed in fixed cells in a 2 dimensional grid and it is not possible for tiles to overlap. This strategy is implemented with a genetic algorithm. In the second strategy, which is implemented using genetic programming, the tiles can be placed in any position and at an arbitrary rotation. It is possible for one tile to be placed on top of another so a method for dealing with overlap is needed. We have investigated three methods for dealing with overlap. The second strategy generates more engaging animations but at considerably increased computational cost. We conclude that evolutionary search can be used to produce very engaging animations in which a target image gradually emerges from an initial random collection of tiles.

Detecting whether computer program code is a student's original work or has been copied from another student or some other source is a major problem for many universities. Detection methods based on the information retrieval concepts of indexing and similarity matching scale well to large collections of files, but require appropriate similarity functions for good performance. We have used particle swarm optimization and genetic programming to evolve similarity functions that are suited to computer program code. Using a training set of plagiarised and non-plagiarised programs we have evolved better parameter values for the previously published Okapi BM25 similarity function. We have then used genetic programming to evolve completely new similarity functions that do not conform to any predetermined structure. We found that the evolved similarity functions outperformed the human developed Okapi BM25 function. We also found that a detection system using the evolved functions was more accurate than the the best code plagiarism detection system in use today, and scales much better to large collections of files. The evolutionary computing techniques have been extremely useful in finding similarity functions that advance the state of the art in code plagiarism detection.

We investigated how indexed FOR-loops, such as the ones found in procedural programming languages, can be implemented in genetic programming. We use them to train programs that learn the repeating unit string of a given regular binary pattern string and can reproduce the learnt pattern to an arbitrary size, specified by a parameter N. We discovered that this particular problem, where the solution needs to scale with multiple size-instances of the problem, is very hard to solve without the help of domain knowledge.

This paper describes a texture segmentation method using genetic programming (GP), which is one of the most powerful evolutionary computation algorithms. By choosing an appropriate representation texture, classifiers can be evolved without computing texture features. Due to the absence of time-consuming feature extraction, the evolved classifiers enable the development of the proposed texture segmentation algorithm. This GP based method can achieve a segmentation speed that is significantly higher than that of conventional methods. This method does not require a human expert to manually construct models for texture feature extraction. In an analysis of the evolved classifiers, it can be seen that these GP classifiers are not arbitrary. Certain textural regularities are captured by these classifiers to discriminate different textures. GP has been shown in this study as a feasible and a powerful approach for texture classification and segmentation, which are generally considered as complex vision tasks.

Loops are rarely used in genetic programming due to issues such as detecting infinite loops and invalid programs. In this paper we present a restricted form of loops that is specifically designed to be evolved in image classifiers. Particularly, we evolve classifiers that use these loops to perform calculations on image regions chosen by the loops. We have compared this method to another classification method that only uses individual pixels in its calculations. These two methods are tested on two synthesised and one non-synthesised greyscale image classification problems of varying difficulty. The most difficult problem requires determining heads or tails of 320 x 320 pixel images of a US one cent coin at any angle. On this problem, the accuracy of the loops approach was 97.80% in contrast to the no-loop method accuracy of 79.46%. Use of loops also reduces overfitting of training data. We also found that loop methods overfit less when only a few training examples are available.

We describe the implementation of an art installation that generates animated photomosaics of the viewer. Photomosaics are target images composed of smaller images known as tiles. When a photomosaic is viewed from afar the detail of the tiles is lost and the target image is evident. Up close, the opposite occurs: the detail of the tiles is evident and the target image is lost. Our system uses a photo of the viewer as the target and miniatures of the viewer's face as tiles. Evolutionary search is used to find the best selection and arrangement of tiles. Each newly found best image is then used as the frame of a movie. The resulting animations start from a random arrangement of tiles and gradually the viewer's face emerges and is clearly visible, and then gradually de-materialises into a random pattern.

We describe a progression of solutions to a di ffi cult object detection problem, that of locating landmarks in X-Rays used in orthodontic treatment planning. In our fi rst formula tion an object detector was a genetic program whose inputs were a number of attributes computed from a scanning window. We used a rich function set comprising f+; ; -; ÷ ; min; max; ifthenelseg. Experimentation with di ff erent function sets revealed that using the function set f+; g gave detectors that were almost as accurate. Such detectors are essentially a linear combination of attributes so we also implemented a parameter optimization solution with a particle swarm optimizer. Contrary to expectation, the PSO detectors are more accurate and smaller than the GP ones. Our analysis of the reasons for this reveals that (1) the PSO approach involves a considerably smaller search space than the GP approach, (2) in the PSO approach there is a 1-1 mapping between genotype and phenotype while in the GP approach this mapping is many-1 and many semantically equivalent potential solutions are evaluated, (3) the fi tness landscape for PSO is a good one for search in that solutions are distributed in areas of high fi tness that are easy to locate while the GP landscape is much more di ffi cult to characterize and areas of high fi tness more difficult to find.

Learning goal-scoring behaviour from scratch for simulated robot soccer is considered to be a very difficult problem, and is often achieved by endowing players with an innate set of hand-coded skills, or by decomposing the problem into learning a set of simpler behaviours which are then aggregated into goal-scoring behaviour. When only basic skills are available to the player the fitness landscape is very flat, containing only a few thin peaks. As more human expertise is injected via hand-coded skills or a composite fitness function, more gradient information becomes apparent on the landscape and the genetic search is more successful. The work presented in this paper uses autocorrelation and information content measures to examine features of the fitness landscape to explain how the difficulty of the problem is changed by injecting human expertise.

This paper describes an approach to evolving texture feature extraction programs using tree based genetic programming. The programs are evolved from a learning set of 13 textures selected from the Brodatz database. In the evolutionary phase, texture images are first "binarised" to 256 grey levels. An encoding of the positions of the black pixels is used as the input to the evolved programs. A separate feature extraction program is evolved for each of the 256 grey levels. Fitness is measured by applying the evolved program to all of the images in the learning set, using one dimensional clustering on the outputs and then using the separation between the clusters as the fitness value. On two benchmark problems using the evolved programs for feature extraction and a nearest neighbour classifier, the evolved features gave test accuracies of 74.6% and 66.2% respectively for a 13 Brodatz and a 15 Vistex texture problem. This is better than a number of human derived methods on the same problems.

In this paper we analyse the reasons why evolving programs with a restricted form of loops is superior to evolving programs without loops for two problems which have underlying repetitive characteristics - a visit-every-square problem and a modified Santa Fe ant problem. We show that in the case of loops there is a larger number of solutions with smaller tree sizes. We show that the computational patterns captured in the bodies of the loops are reflective of repeating patterns in the domain. We show that the increased computational cost of evaluating an individual can be controlled by domain knowledge.

The application of multi-objective evolutionary computation techniques to the genetic programming of classifiers has the potential to both improve the accuracy and decrease the training time of the classifiers. The performance of two such algorithms is investigated on the even 6-parity problem and the Wisconsin breast cancer, Iris and Wine data sets from the UCI repository. The first method explores the addition of an explicit size objective as a parsimony enforcement technique. The second represents a program's classification accuracy on each class as a separate objective. Both techniques give a lower error rate with less computational cost than was achieved using a standard GP with the same parameters.

Layered learning allows decomposition of the stages of learning in a problem domain. We apply this technique to the evolution of goal scoring behavior in soccer players and show that layered learning is able to find solutions comparable to standard genetic programs more reliably. The solutions evolved with layers have a higher accuracy but do not make as many goal attempts. We compared three variations of layered learning and find that maintaining the population between layers as the encapsulated learnt layer is introduced to be the most computationally efficient. The quality of solutions found by layered learning did not exceed those of standard genetic programming in terms of goal scoring ability.

Program Bloat - phenomenon of ever-increasing program size during a GP run - is a recognised and widespread problem. Traditional techniques to combat program bloat are program size limitations of parsimony pressure (penalty functions). These techniques suffer from a number of problems, in particular their reliance on parameters whose optimal values it is difficult to a priori determine. In this paper, we introduce POPE-GP, a system that makes use of the NSGA-II multiobjective evolutionary algorithm as an alternative, parameter-free technique for eliminating program bloat. We test it on a classification problem and find that while vastly reducing program size, it does improve generalisation performance.

Evolving programs with explicit loops presents major difficulties, primarily due to the massive increase in the size of the search space. Fitness evaluation becomes computationally expensive and a method for dealing with infinite loops must be implemented. We have investigated ways of dealing with these problems by the evolution of for-loops of increasing semantic complexity. We have chosen two problems - a modified Santa Fe ant problem and a sorting problem - which have natural looping constructs in their solution and a solution without loops is not possible unless the tree depth is very large. We have shown that by controlling the complexity of the loop structures it is possible to evolve smaller and more understandable programs for these problems.

This work presents the use of genetic programming (GP) to a complex domain, texture analysis. Two major tasks of texture analysis, texture classification and texture segmentation, are studied. Bitmap textures are used in this investigation. In classification tasks, the results show that GP is able to evolve accurate classifiers based on texture features. Moreover by using the presented method, GP is able to evolve accurate classifiers without extracting texture features. In texture segmentation tasks, the investigation shows that a fast and accurate segmentation method can be developed based on GP generated texture classifiers. Our further investigation show that the accuracies are not achieved by chance. There are regularities been captured by GP-generated classifiers in performing texture discrimination.

This paper describes a domain independent approach to multiple class rotation invariant 2D object detection problems. The approach avoids preprocessing, segmentation and specific feature extraction. Instead, raw image pixel values are used as inputs to the learning systems. Five object detection methods have been developed and tested, the basic method and four variations which are expected to improve the accuracy of the basic method. In the basic method cutouts of the objects of interest are used to train multilayer feed forward networks using back propagation. The trained network is then used as a template to sweep the full image and find the objects of interest. The variations are (1) Use of a centred weight initialization method in network training, (2) Use of a genetic algorithm to train the network, (3) Use of a genetic algorithm, with fitness based on detection rate and false alarm rate, to refine the weights found in basic approach, and (4) Use of the same genetic algorithm to refine the weights found by method 2. These methods have been tested on three detection problems of increasing difficulty: an easy database of circles and squares, a medium difficulty database of coins and a very difficult database of retinal pathologies. For detecting the objects in all classes of interest in the easy and the medium difficulty problems, a 100% detection rate with no false alarms was achieved. However the results on the retinal pathologies were unsatisfactory. The centred weight initialization algorithm improved the detection performance over the basic approach on all three databases. In addition, refinement of weights with a genetic algorithm significantly improved detection performance on the three databases. The goal of domain independent object recognition was achieved for the detection of relatively small regular objects in larger images with relatively uncluttered backgrounds. Detection performance on irregular objects in complex, highly cluttered backgrounds such as the retina pictures, however, has not been achieved to an acceptable level.

In deceptive problems many runs lead to suboptimal solutions and it can be difficult to escape from these local optima and find the global best solution. We propose a pyramid search strategy for these kinds of problems. In the pyramid strategy a number of populations are initialised and independently evolved for a number of generations at which point the worst performing populations are discarded. This evolve/discard process is continued until the problem is solved or one population remains. We show that for a number of deceptive problems the pyramid strategy results in a higher probability of success with fewer evaluations and a lower standard deviation of the number evaluations to success than the conventional approach of running to a maximum number of generations and then restarting.

This paper presents a method which extends the use of genetic programming (GP) to a complex domain, texture segmentation. By this method, segmentation tasks are performed by texture classifiers which are evolved by the GP. Small cutouts sampled from images of various textures are used for the evolution. The generated classifiers directly use pixel values as input. Based on these classifiers an algorithm which uses a voting strategy to partition texture regions is developed. The results of the investigation indicate that the proposed method is able to accurately identify the boundaries between different texture regions, even if the boundaries are not regular. The method can segment two textures as well as multiple textures. Furthermore, fast segmentation can be achieved. The speed of the proposed texture segmentation method can be a hundred times faster than conventional methods.

We have investigated an approach to preventing or minimising the occurrence of premature convergence by measuring the similarity between the programs in the population and replacing the most similar ones with randomly generated programs. On a problem with known premature convergence behaviour, the MAX problem, similarity replacement significantly decreased the rate of premature convergence over the best that could be achieved by manipulation of the mutation rate. The expected CPU time for a successful run was increased due to the additional cost of the similarity matching. On a problem which has a very expensive fitness function, the evolution of a team of soccer playing programs, the degree of premature convergence rate was also significantly reduced. However, in this case the expected time for a successful run was significantly decreased indicating that similarity replacement can be worthwhile for problems with expensive evaluation functions. A significant discovery from our experimental work is that a small change to the way mutation is carried out can result in significant reductions in premature convergence.

We investigate the behaviour of image texture classifiers generated by genetic programming. We propose techniques to understand how classifiers capture textural characteristics and for discussing the effectiveness of different classifiers. Our results show that regularities of patterns can be detected by the genetic programming method without predefined knowledge.

Five alternative methods are proposed to perform multi-class classification tasks using genetic programming. These methods are: (1) binary decomposition, in which the problem is decomposed into a set of binary problems and standard genetic programming methods are applied; (2) static range selection, where the set of real values returned by a genetic program is divided into class boundaries using arbitrarily-chosen division points; (3) dynamic range selection, in which a subset of training examples are used to determine where, over the set of reals, class boundaries lie; (4) class enumeration, which constructs programs similar in syntactic structure to a decision tree; and (5) evidence accumulation, which allows separate branches of the program to add to the certainty of any given class. The results show that the dynamic range selection method is well-suited to the task of multi-class classification and is capable of producing classifiers that are more accurate than the other methods tried when comparable training times are allowed. The accuracy of the generated classifiers was comparable to alternative approaches over several data sets.

We describe an algorithm animation system for artificial intelligence search algorithms. We have identified a number learning difficulties experienced by students studying search algorithms and designed the animation system to help students to overcome these difficulties. As well as the usual single step mode for assistance in learning the individual steps of an algorithm, the system supports an innovative burst mode for visualising qualitative behaviour and facilitating comparisons between different algorithms and heuristic functions. The system has successfully been used in the classroom for 4 years and survey results indicate use of the system improves understanding. An empirical study comparing a group of 15 students using the animation system and 15 students who wrote programs for the algorithms revealed a generally similar level of understanding, however the animation group was better at dealing with questions about qualitative behaviour.

This paper describes a method of utilising genetic algorithms to train fixed architecture feedforward and recurrent neural networks. The technique described uses the genetic algorithm to evolve changes to the weights and biases of the network rather than the weights and biases themselves. Results achieved by this technique indicate that for many problems it compares very favourably with the more common gradient descent techniques for training neural networks, and in some cases is superior. The technique is useful for those problem which are known to be difficult for the gradient descent techniques.

A "Front End" for a Computer-Based Glaucoma Consultation System is described. The system views a case as a description of a particular instance of a class of concepts called "structured objects" and builds up a representation of the instance from the sentences in the case. the information required by the consultation system is then extracted and passed on to the consultation system in the appropriately coded form. A core of syntactic, semantic and contextual rules which are applicable to all structured objects is being developed together with a representation of the structured object GLAUCOMA-PATIENT. There is also a facility for adding domain dependent syntax, abbreviations and defaults.