Perceptual aliasing, where distinct states appear indistinguishable due to sensor limitations or environmental ambiguities, poses significant challenges in multi-step decision-making. The eXtended Classifier System (XCS) addresses this issue by identifying unique state transition patterns and combining them to construct accurate policies. Additionally, state-action-state chains enhance XCS's ability to handle sequentially aliased states. However, XCS processes aliased states sequentially as they are perceived, which can lead to performance degradation when incorrect versions of aliased states are included in the chain. This limitation underscores the need for a more robust mechanism to accurately differentiate unique states from aliased ones to ensure reliable policy creation. To address this, we propose a dual-stream identification framework that enhances XCS's performance in environments with perceptual aliasing. The framework introduces two parallel identification processes: one captures immediate state-action relationships, while the other identifies broader patterns across multi-step sequences. By integrating these dual streams, the proposed approach effectively disambiguates aliased states, enabling more accurate decision-making. Experimental evaluations demonstrate that our dual-stream model outperforms state-of-the-art XCS implementations across 14 benchmark environments.

A domain shift exists between the large-scale, internet data used to train a Vision-Language Model (VLM) and the raw image streams collected by a robot. Existing adaptation strategies require the definition of a closed-set of classes, which is impractical for a robot that must respond to diverse natural language queries. In response, we present QueryAdapter; a novel framework for rapidly adapting a pre-trained VLM in response to a natural language query. QueryAdapter leverages unlabelled data collected during previous deployments to align VLM features with semantic classes related to the query. By optimising learnable prompt tokens and actively selecting objects for training, an adapted model can be produced in a matter of minutes. We also explore how objects unrelated to the query should be dealt with when using real-world data for adaptation. In turn, we propose the use of object captions as negative class labels, helping to produce better calibrated confidence scores during adaptation. Extensive experiments on ScanNet++ demonstrate that QueryAdapter significantly enhances object retrieval performance compared to state-of-the-art unsupervised VLM adapters and 3D scene graph methods. Furthermore, the approach exhibits robust generalization to abstract affordance queries and other datasets, such as Ego4D.

Deep-learning and large scale language-image training have produced image object detectors that generalise well to diverse environments and semantic classes. However, existing object detection paradigms are not optimally tailored for the embodied conditions inherent in robotics, where the same objects are repeatedly observed over time. In this setting, detectors that operate on single images or short sequences are likely to produce inconsistent predictions. Motivated by this, we explore if the embodiment of the detector can be utilised to generate more consistent and reliable detections during repeat observation of a scene. We propose a novel framework that incrementally updates a spatial feature memory while using it as a prior to perform image object detection. By leveraging the embodiment of the robot in this way, raw object detection performance is enhanced by up to 4.12 mAP and downstream robotic tasks such as semantic mapping and object recall are improved. We also investigate the structure this spatial memory should take, leading to an implementation that aggregates features from the shared language-image embedding space. This approach allows the detector to effectively balance the use of memory and image features, while ensuring that the benefits of language-image pre-training can be enjoyed alongside our spatial memory.

Evolutionary Computation (EC) often throws away learned knowledge as it is reset for each new problem addressed. Conversely, humans can learn from small-scale problems, retain this knowledge (plus functionality), and then successfully reuse them in larger-scale and/or related problems. Linking solutions to problems has been achieved through layered learning, where an experimenter sets a series of simpler related problems to solve a more complex task. Recent works on Learning Classifier Systems (LCSs) has shown that knowledge reuse through the adoption of Code Fragments, GP-like tree-based programs, is plausible. However, random reuse is inefficient. Thus, the research question is how LCS can adopt a layered-learning framework, such that increasingly complex problems can be solved efficiently. An LCS (named XCSCF*) has been developed to include the required base axioms necessary for learning, refined methods for transfer learning and learning recast as a decomposition into a series of subordinate problems. These subordinate problems can be set as a curriculum by a teacher, but this does not mean that an agent can learn from it; especially if it only extracts over-fitted knowledge of each problem rather than the underlying scalable patterns and functions. Results show that from a conventional tabula rasa, with only a vague notion of which subordinate problems might be relevant, XCSCF* captures the general logic behind the tested domains and therefore can solve any n-bit Multiplexer, n-bit Carry-one, n-bit Majority-on, and n-bit Even-parity problems. This work demonstrates a step towards continual learning as learned knowledge is effectively reused in subsequent problems.

Over the past four decades, Learning Classifier Systems (LCSs) have faced challenges in producing accurate and interpretable models for domains with continuous features, mainly due to the irrelevance issue caused by genotypic methods. These methods directly modify genotypes (conditions), leading to the creation of irrelevant rules. Phenotypic LCSs, which first modify a rule's phenotype (covered instance set) before altering its genotype, can avoid this issue. However, previous phenotypic LCSs struggle with overfitting, resulting in lower testing performance. In response, we propose a novel phenotypic LCS featuring innovations: 1) a heterogeneous phenotype approach in the rule discovery mechanism to alleviate overfitting, and 2) Informed Mutation leverages the inherent neighbouring of similar instances to enhance rule generalization, thereby improving model interpretability. The proposed LCS demonstrates its success with superior testing performance and more interpretable models in all experiments compared to other LCSs. Notably, in a problem with 2048 features, the proposed LCS model outperformed the genotypic UCS by achieving a 97.4% testing accuracy with just 13 rules, compared to the UCS's 9961 rules but only 49.9% accuracy.

Counterfactual explanation is a popular eXplainable AI technique, that gives contrastive explanations to answer potential "what-if" questions about the workings of machine learning models. However, research into how explanations are understood by human beings has shown that an optimal explanation should be both selected and social, providing multiple varying explanations for the same event that allow a user to select specific explanations based on prior beliefs and cognitive biases. In order to provide such explanations, a Rashomon set of explanations can be created: a set of explanations utilising different features in the data. Current work to generate counterfactual explanations does not take this need into account, only focusing on producing a single optimal counterfactual.

Modern classifier systems can effectively classify targets that consist of simple patterns. However, they can fail to detect hierarchical patterns of features that exist in many real-world problems, such as understanding speech or recognizing object ontologies. Biological nervous systems have the ability to abstract knowledge from simple and small-scale problems in order to then apply it to resolve more complex problems in similar and related domains. It is thought that lateral asymmetry of biological brains allows modular learning to occur at different levels of abstraction, which can then be transferred between tasks. This work develops a novel evolutionary machine-learning (EML) system that incorporates lateralization and modular learning at different levels of abstraction. The results of analyzable Boolean tasks show that the lateralized system has the ability to encapsulate underlying knowledge patterns in the form of building blocks of knowledge (BBK). Lateralized abstraction transforms complex problems into simple ones by reusing general patterns (e.g., any parity problem becomes a sequence of the 2-bit parity problem). By enabling abstraction in evolutionary computation, the lateralized system is able to identify complex patterns (e.g., in hierarchical multiplexer (HMux) problems) better than existing systems.

Human intelligence can simultaneously process many tasks with the ability to accumulate and reuse knowledge. Recent advances in artificial intelligence, such as transfer, multitask, and layered learning, seek to replicate these abilities. However, humans must specify the task order, which is often difficult particularly with uncertain domain knowledge. This work introduces a continual-learning system (ConCS), such that given an open-ended set of problems once each is solved its solution can contribute to solving further problems. The hypothesis is that the evolutionary computation approach of learning classifier systems (LCSs) can form this system due to its niched, cooperative rules. A collaboration of parallel LCSs identifies sets of patterns linking features to classes that can be reused in related problems automatically. Results from distinct Boolean and integer classification problems, with varying interrelations, show that by combining knowledge from simple problems, complex problems can be solved at increasing scales. 100% accuracy is achieved for the problems tested regardless of the order of task presentation. This includes intractable problems for previous approaches, e.g.,$n$-bit Majority-on. A major contribution is that human guidance is now unnecessary to determine the task learning order. Furthermore, the system automatically generates the curricula for learning the most difficult tasks.

Counterfactual explanations are a popular eXplainable AI technique, used to provide contrastive answers to “what-if” questions. These explanations are consistent with the way that an everyday person will explain an event, and have been shown to satisfy the ‘right to explanation’ of the European data regulations. Despite this, current work to generate counterfactual explanations either makes assumptions about the model being explained or utlises algorithms that perform suboptimally on continuous data. This work presents two novel algorithms to generate counterfactual explanations using Particle Swarm Optimization (PSO) and Differential Evolution (DE). These are shown to provide effective post-hoc explanations that make no assumptions about the underlying model or data structure. In particular, PSO is shown to generate counterfactual explanations that utilise significantly fewer features to generate sparser explanations when compared to previous related work.

Interest in reinforcement learning (RL) has recently surged due to the application of deep learning techniques, but these connectionist approaches are opaque compared with symbolic systems. Learning Classifier Systems (LCSs) are evolutionary machine learning systems that can be categorised as eXplainable AI (XAI) due to their rule-based nature. Michigan LCSs are commonly used in RL domains as the alternative Pittsburgh systems (e.g. SAMUEL) suffer from complex algorithmic design and high computational requirements; however they can produce more compact/interpretable solutions than Michigan systems. We aim to develop two novel Pittsburgh LCSs to address RL domains: PPL-DL and PPL-ST. The former acts as a "zeroth-level" system, and the latter revisits SAMUEL's core Monte Carlo learning mechanism for estimating rule strength. We compare our two Pittsburgh systems to the Michigan system XCS across deterministic and stochastic FrozenLake environments. Results show that PPL-ST performs on-par or better than PPL-DL and outperforms XCS in the presence of high levels of environmental uncertainty. Rulesets evolved by PPL-ST can achieve higher performance than those evolved by XCS, but in a more parsimonious and therefore more interpretable fashion, albeit with higher computational cost. This indicates that PPL-ST is an LCS well-suited to producing explainable policies in RL domains.

Multipoint dynamic aggregation is a meaningful optimization problem due to its important real-world applications, such as post-disaster relief, medical resource scheduling, and bushfire elimination. The problem aims to design the optimal plan for a set of robots to execute geographically distributed tasks. Unlike the majority of scheduling and routing problems, the tasks in this problem can be executed by multiple robots collaboratively. Meanwhile, the demand of each task changes over time at an incremental rate and is affected by the abilities of the robots executing it. This poses extra challenges to the problem, as it has to consider complex coupled relationships among robots and tasks. To effectively solve the problem, this article develops a new metaheuristic algorithm, called adaptive coordination ant colony optimization (ACO). We develop a novel coordinated solution construction process using multiple ants and pheromone matrices (each robot/ant forages a path according to its own pheromone matrix) to effectively handle the collaborations between robots. We also propose adaptive heuristic information based on domain knowledge to promote efficiency, a pheromone-based repair mechanism to tackle the tight constraints of the problem, and an elaborate local search to enhance the exploitation ability of the algorithm. The experimental results show that the proposed adaptive coordination ACO significantly outperforms the state-of-the-art methods in terms of both effectiveness and efficiency.

The multipoint dynamic aggregation (MPDA) problem of the multirobot system is of great significance for its real-world applications such as bush fire elimination. The problem is to design the optimal plan for a set of heterogeneous robots to complete some geographically distributed tasks collaboratively. In this article, we consider the dynamic version of the problem, where new tasks keep appearing after the robots are dispatched from the depot. The dynamic MPDA problem is a complicated optimization problem due to several characteristics, such as the collaboration of robots, the accumulative task demand, the relationships among robots and tasks, and the unpredictable task arrivals. In this article, a new model of the problem considering these characteristics is proposed. To solve the problem, we develop a new genetic programming hyperheuristic (GPHH) method to evolve reactive coordination strategies (RCSs), which can guide the robots to make decisions in real time. The proposed GPHH method contains a newly designed effective RCS heuristic template to generate the execution plan for the robots according to a GP tree. A new terminal set of features related to both robots and tasks and a cluster filter that assigns the robots to urgent tasks are designed. The experimental results show that the proposed GPHH significantly outperformed the state-of-the-art methods. Through further analysis, useful insights such as how to distribute and coordinate robots to execute different types of tasks are discovered.

Perceptual aliasing challenges reinforcement learning agents. They struggle to learn stable policies by failing to identify and disambiguate perceptually identical states in the environment that require different actions to reach a goal. As the agent often has only a local frame of reference, it cannot represent the global environment. Frame-of-reference-based learning is a feature of vertebrate intelligence that allows multiple simultaneous representations of an environment at different levels of abstraction. This enables the resolution of patterns that are made up of patterns that are made up of features. The evolutionary computation technique of learning classifier systems has shown promise in learning nested patterns in single-step domains. This work uses the frame-of-reference concept within a learning classifier system to learn stable policies in non-Markov multistep domains. Considering aliased states at a constituent level enables the system to place them appropriately in holistic-level policies. Instead of enumerating a huge search space, evolution computation empowers the novel system to evolve fitter rules and policies. The experimental results show that the novel system effectively solves complex aliasing patterns in non-Markov environments that have been challenging to artificial agents. For example, the novel system utilizes only 6.5, 3.71, and 3.22 steps to resolve Maze10, Littman57, and Woods102, respectively.

A major goal of machine learning is to create techniques that abstract away irrelevant information. The generalisation property of standard Learning Classifier Systems (LCSs) removes such information at the feature level but not at the feature interaction level. Code Fragments (CFs), a form of tree-based programs, introduced feature manipulation to discover important interactions, but they often contain irrelevant information, which causes structural inefficiency. XOF is a recently introduced LCS that uses CFs to encode building blocks of knowledge about feature interaction. This paper aims to optimise the structural efficiency of CFs in XOF. We propose two measures to improve constructing CFs to achieve this goal. Firstly, a new CF-fitness update estimates the applicability of CFs to the problem while also considering the structural complexity. The second measure is a niche-based method for generating CFs. These approaches were tested on Even-parity and Hierarchical problems, which require highly complex combinations of input features to capture the data patterns. The results show that the proposed methods significantly increase the structural efficiency of CFs, which is estimated by the rule "generality rate". This results in faster learning performance in the Hierarchical Majority-on problem. Furthermore, a user-set depth limit for CF generation is not needed as the learning agent will not adopt higher-level CFs once optimal CFs are constructed.

Emotion categorization plays an important role in understanding human emotions by artificial intelligence systems such as robots. It is a difficult task as humans express many features, which vary over time when showing an emotion. Thus, existing classification techniques are overwhelmed, and the creation of a subset of appropriate features is needed. Feature selection can be used to improve the performance of an emotion categorization task by selecting a subset of features. This removes irrelevant features. Particle swarm optimization (PSO) is a meta-heuristic algorithm which has demonstrated excellent performance in feature selection tasks. However, traditional PSO algorithms often get trapped in local optima as they use their personal best and global best to determine their search direction, which may lead to premature convergence. In this paper, we present a time-based PSO variant by introducing a time-constant into the velocity update function of the PSO algorithm to avoid premature convergence, particularly in an emotion video-frame dataset. The method has been incorporated into binary and continuous PSO, then compared with the two standard versions on an emotion video-frame (CK+) dataset, as well as on static emotional datasets (i.e. the JAFFE and NIMH-ChEFS) to ensure that bias has not been introduced into the algorithm. While the time-based PSO variant (both binary and the continuous PSO) have achieved non-significantly higher performance than the standard PSO algorithms on the JAFFE (77.15% vs 75.61%) and NIMH-ChEFS (71.57% vs 70.53%) dataset, the performance is significantly higher on the CK+ (96.19% vs 94.06%) dataset.

A robust solution to the optimal micro-grid (MG) sizing problem requires comprehensive quantification of the underlying parametric uncertainties - particularly, the uncertainty in forecasts of meteorological, load demand, and wholesale electricity price time-series data. However, the associated data-driven processes for probabilistic uncertainty quantification are computationally expensive. Accordingly, the mainstream meta-heuristic-based MG sizing literature has failed to concurrently quantify more than four sources of forecast uncertainty. To address this knowledge gap, this paper introduces a novel computationally efficient, probabilistic MG sizing model that enables the simultaneous treatment of any (reasonable) number of data uncertainty. This provides a platform to characterize the uncertainty in ambient temperature and river streamflow for the first time in the MG optimal sizing literature. Importantly, the model supports the associated long-term strategic MG energy planning optimization processes through in-depth analyses of the worst-case, most likely case, and best-case planning scenarios. To demonstrate the utility of the proposed model for community MG projects, a case study is presented for the town of Ohakune, New Zealand. Notably, the numeric simulation results have shown that the whole-life cost of the conceptualized MG would have been underestimated and overestimated by as much as ~17% and ~30% respectively in the best-case and worst-case scenarios if the problem-inherent uncertainties were not explicitly factored into the associated techno-economic analyses.

Evolutionary computation has brought great progress to rule-based learning but this progress is often blind to the optimality of the system design. This article theoretically reveals an optimal learning scheme on the most popular evolutionary rule-based learning approach-the accuracy-based classifier system (or XCS). XCS seeks to form accurate, maximally general rules that together classify the state space of a given domain. Previously, setting up the system to perform well has been a “blackart” as no systematic approach to XCS parameter tuning existed. We derive a theoretical approach that mathematically guarantees that XCS identifies the accurate rules, which also returns a theoretically valid XCS parameter setting. Then, we demonstrate our theoretical setting derives the maximum correctness of rule-identification in the fewest iterations possible. We also experimentally show that our theoretical setting enables XCS to easily solve several challenging problems where it had previously struggled.

Learning Classifier Systems (LCSs) are a paradigm of rule-based evolutionary computation (EC). LCSs excel in data-mining tasks regarding helping humans to understand the explored problem, often through visualizing the discovered patterns linking features to classes. Due to the stochastic nature of EC, LCSs unavoidably produce and keep redundant rules, which obscure the patterns. Thus, rule compaction methods are invoked to produce a better population by removing problematic rules. Previously, compaction methods have neither been tested on large-scale problems nor been assessed on the performance of capturing patterns. We review and test the most popular compaction algorithms, finding that across multiple LCSs’ populations for the same task, although the redundant rules can be different, the accurate rules are common. Furthermore, the patterns contained consistently refer to the nature of the explored domain, e.g., the data distribution or the importance of features for determining actions. This extends the [ O ] set hypothesis proposed by Butz et al. [1], in which an LCS is expected to evolve a minimal number of non-overlapped rules to represent an addressed domain. Two new compaction algorithms are introduced to search at the rule level and the population level by compacting multiple LCSs’ populations. Two visualization methods are employed for verifying the interpretability of these populations. Successful compaction is demonstrated on complex and real problems with clean datasets, e.g., the 11-bits Majority-On problem that requires 924 different interacting rules in the optimal solution to be uniquely identified to enable correct visualization. For the first time, the patterns contained in learned models for the large-scale 70-bits Multiplexer problem are visualized successfully.

Multi-Point Dynamic Aggregation (MPDA) is a novel task model to determine task allocation for a multi-robot system. In an MPDA scenario, several robots with different abilities aim to complete a set of tasks cooperatively. The demand of each task is time varying. It increases over time at a certain rate (e.g. the bush fire in Australia). When a robot executes a task, the demand of the task decreases at another certain rate, depending on the robot's ability. In this paper, the objective is to design a task plan for minimising the maximal completed time of all tasks. But coupling cooperative and time-varying characteristics of MPDA brings great challenges to modelling, decoding, and optimisation. In this paper, a multi-permutation encoding is used to represent every robot's visiting sequence of tasks, and an implicit decoding strategy with heuristic rules is designed to simplify the problem from a hybrid variable optimisation to a multi-permutation optimisation. Memetic algorithms for the task allocation of MPDA with two local search methods are designed: equality one-step local search with a better exploration ability and elite multi-step local search with a better exploitation ability. Computational experiments show that the proposed decoding method leads to a better performance given the same computational time budget. Experimental results also show that the proposed memetic algorithms outperform the state-of-the-art method in solving the task planning problems of MPDA.

Learning Classifier Systems (LCSs) are a group of rule-based evolutionary computation techniques, which have been frequently applied to data-mining tasks. Evidence shows that LCSs can produce models containing human-discernible patterns. But, traditional LCSs cannot efficiently discover consistent, general rules - especially in domains that have unbalanced class distribution. The reason is that traditional search methods, e.g. crossover, mutation, and roulette wheel deletion, rely on stochasticity to find and keep optimum rules. Recently, absumption has been introduced to deterministically remove over-general rules, which is complementary to subsumption that deterministically removes over-specific rules. It is hypothesized that utilizing just assumption & subsumption transforms the search process from stochastic to deterministic, which benefits LCSs in evolving interpretable models and removing the need to tune search parameters to the problem. Interrogatable artificial Boolean domains with varying numbers of attributes are considered as benchmarks. The new LCS, termed Absumption Subsumption Classifier System (ASCS), successfully produces interpretable models for all the complex domains tested, whereas the non-optimal rules in existing techniques obscure the patterns. ACSC's ability to handle complex search spaces is observed, e.g. for the 14-bits Majority-On problem the required 6435 different cooperating rules were discovered enabling correct pattern visualization.

Multitask Learning is a learning paradigm that deals with multiple different tasks in parallel and transfers knowledge among them. XOF, a Learning Classifier System using tree-based programs to encode building blocks (meta-features), constructs and collects features with rich discriminative information for classification tasks in an Observed List. This paper seeks to facilitate the automation of feature transferring in between tasks by utilising the Observed List. We hypothesise that the best discriminative features of a classification task carry its characteristics. Therefore, the relatedness between any two tasks can be estimated by comparing their most appropriate patterns. We propose a multiple-XOF system, called mXOF, that can dynamically adapt feature transfer among XOFs. This system utilises the Observed List to estimate the task relatedness. This method enables the automation of transferring features. In terms of knowledge discovery, the resemblance estimation provides insightful relations among multiple data. We experimented mXOF on various scenarios, e.g. representative Hierarchical Boolean problems, classification of distinct classes in the UCI Zoo dataset, and unrelated tasks, to validate its abilities of automatic knowledgetransfer and estimating task relatedness. Results show that mXOF can estimate the relatedness reasonably between multiple tasks to aid the learning performance with the dynamic feature transferring.

Deep learning is an important field of machine learning. It is playing a critical role in a variety of applications ranging from self-driving cars to security and surveillance. However, deep networks have deep flaws. For example, they are highly vulnerable to adversarial attacks. One reason may be the homogeneous nature of their knowledge representation, which allows a single disruptive pattern to cause miss-classification. Biological intelligence has lateral asymmetry, which allows heterogeneous, modular learning at different levels of abstraction, enabling different representations of the same object.

Emotion recognition has become an increasingly important area of research due to the increasing number of CCTV cameras in the past few years. Deep network-based methods have made impressive progress in performing emotion recognition-based tasks, achieving high performance on many datasets and their related competitions such as the ImageNet challenge. However, deep networks are vulnerable to adversarial attacks. Due to their homogeneous representation of knowledge across all images, a small change to the input image made by an adversary might result in a large decrease in the accuracy of the algorithm. By detecting heterogeneous facial landmarks using the machine learning library Dlib we hypothesize we can build robustness to adversarial attacks. The residual neural network (ResNet) model has been used as an example of a deep learning model. While the accuracy achieved by ResNet showed a decrease of up to 22%, our proposed approach has shown strong resistance to an attack and showed only a little (<; 0.3%) or no decrease when the attack is launched on the data. Furthermore, the proposed approach has shown considerably less execution time compared to the ResNet model.

Code Fragments (CFs) are a new representation for classifier conditions in Learning Classifier Systems (LCSs). CFs are Genetic Programming-like trees that use functions as internal nodes, and data input or previously learned CFs as leaf nodes for feature construction. The XCSCFC system used CFs in rule conditions of XCS, an accuracy-based Michigan-style LCS, to transfer knowledge and thus solve large-scale problems. However, the trade-off for the richness and flexibility that allows CFs to compactly describe decision boundaries results in an undesired increase in the search space of solutions. Therefore, this paper proposes a novel model extension for Online Feature-generation (OF), which enables evolving features (CFs) through an online observed list of CFs. This extension enables a method of estimating the worth of CFs to identifying the patterns in the problem in order to construct applicable high-level features. The experiments show that the XCS with OF (XOF) can solve the benchmark problems in fewer generations compared with XCSCFC in non-transfer learning scenarios. The novel search of CFs successfully built high-level features, which show the rules produced by XOF to be more generalised than previously possible. Consequently, the final solutions contain fewer rules to solve problems as they encode more compact and comprehensive decision boundaries.

A reward mechanism is critical for a Reinforcement Learning agent to learn action policies from rewards. The reward mechanism establishes a policy by estimating contributions of constituents of the policy to a reward. Traditionally, rewards from an environment have two categories: long-term rewards for guiding the policy learning process, and short-term rewards for optimisation. However, long-term, positive rewards are scarce at the initial learning phase in multistep problems such that existing reward mechanisms lack sufficient stimulus to learn policies effectively. This paper proposes XCSCR, an Accuracy-based Learning Classifier System (XCS) algorithm with a combined reward (CR) method, to guide the search for global optimal policies in multistep maze problems. The XCSCR discriminates long-term and short-term rewards through four novel rewardassignment mechanisms: 1) A short-term reward mechanism encourages exploration of the RL agent searching for policies based on short-term rewards. 2) An imprinting mechanism amends the negative impact of indiscriminate rewards between exploration and exploitation. 3) A learning-rate switching mechanism emphasises the impact of long-term positive rewards in the policy searching process. 4) A learning step-threshold mechanism creates an optimisation pressure for policies. Experiments were conducted in three maze environments as this enabled the effects of XCSCR on policies to interpreted easily. Results show that the XCSCR enables learning the optimum path-finding policies quicker and more often than previous XCS algorithms. The XCSCR's improvements for the policy search will facilitate realworld applications, e.g. robotic applications.

Learning Classifier Systems (LCSs), a 40-year-old technique, evolve interrogatable production rules. XCSs are the most popular reinforcement learning based LCSs. It is well established that the subsumption method in XCSs removes overly detailed rules. However, the technique still suffers from overly general rules that reduce accuracy and clarity in the discovered patterns. This adverse impact is especially true for domains that are containing accurate solutions that overlap, i.e. one data instance is covered by two plausible, but competing rules. A novel method, termed absumption, is introduced to counter over-general rules. Complex Boolean problems that contain epistasis, heterogeneity and overlap are used to test the absumption method. Results show that absumption successfully improves the training performance of XCSs by counteracting over-general rules. Moreover, absumption enables the rule-set to be compacted, such that underlying patterns can be precisely visualized successfully. Additionally, the equations for the optimal size of solutions for a problem domain can now be determined.

In complex classification problems, constructed features with rich discriminative information can simplify decision boundaries. Code Fragments (CFs) produce GP-tree-like constructed features that can represent decision boundaries effectively in Learning Classifier Systems (LCSs). But the search space for useful CFs is vast due to this richness in boundary creation, which is impractical. Online Feature-generation (OF) improves the search of useful CFs by growing promising CFs from a dynamic list of preferable CFs based on the ability to produce accurate and generalised, i.e. high-fitness, classifiers. However, the previous preference for high-numerosity CFs did not encapsulate information about the applicability of CFs directly. Consequently, learning performances of OF with an accuracy-based LCS (termed XOF) struggled to progress in the final learning phase. The hypothesis is that estimating the CF-fitness of CFs based on classifier fitness will aid the search for useful constructed features. This is anticipated to drive the search of decision boundaries efficiently, and thereby improve learning performances. Experiments on large-scale and hierarchical Boolean problems show that the proposed systems learn faster than traditional LCSs regarding the number of generations and time consumption. Tests on real-world datasets demonstrate its capability to find readable and useful features to solve practical problems.

Learning Classifier Systems (LCSs) have been widely used to tackle Reinforcement Learning (RL) problems as they have a good generalization ability and provide a simple understandable rule-based solution. The accuracy-based LCS, XCS, has been most popularly used for single-objective RL problems. As many real-world problems exhibit multiple conflicting objectives recent work has sought to adapt XCS to Multi-Objective Reinforcement Learning (MORL) tasks. However, many of these algorithms need large storage or cannot discover the Pareto Optimal solutions. This is due to the complexity of finding a policy having multiple steps to multiple possible objectives. This paper aims to employ a decomposition strategy based on MOEA/D in XCS to approximate complex Pareto Fronts. In order to achieve multi-objective learning, a new MORL algorithm has been developed based on XCS and MOEA/D. The experimental results show that on complex bi-objective maze problems our MORL algorithm is able to learn a group of Pareto optimal solutions for MORL problems without huge storage. Analysis of the learned policies shows successful trade-offs between the distance to the reward versus the amount of reward itself.

Most versions of the XCS Classifier System have been designed to evolve only two rules for each rule discovery invocation, which restricts the search capacity. A difficulty behind generating multiple rules each time is the increase in the probability of deleting immature rules, which conflicts with the requirement that parent rules be sufficiently updated so that fitness represents worth. Thus the aim of this paper is to argue how XCS determines when rules can be deleted safely. The objectives are to certainly identify inaccurate rules and then to maximize how many rules XCS can generate. The proposed method enables adaptation of rule-generation that maximizes the number of generated rules, under the assumption that the reliably inaccurate rules can be replaced with new rules. Experiments show our modification strongly improves the XCS performance on large scale problems, since it can take advantage of multi-point search more efficiently. For example, on the 135-bit multiplexer problem, XCS with our modification requires 1.57 million less training inputs compared with the standard XCS while utilizing the same number of final rules.

XCS is the most popular type of Learning Classifier System, but setting optimum parameter values is more of an art than a science. Early theoretical work required the impractical assumption that classifier parameters had fully converged with infinite update times. The aim of this work is to derive a theoretical condition to mathematically guarantee that XCS identifies maximally accurate classifiers, such that subsequent deletion methods can be used optimally, in as few updates as possible. Consequently, our theory provides a universally usable setup guide for three important parameter settings; the learning rate, the accuracy update and the threshold for subsumption deletion. XCS with our best parameter settings solves the 70-bit multiplexer problem with only 21% of instances that the standard XCS setup needs. On a highly class-imbalanced multiplexer problem with inaccurate classifiers having more than 99.99% classification accuracy, our theory enables XCS to identify only 100% accurate classifiers as accurate and thus obtain the optimal performance.

A main research direction in the field of evolutionary machine learning is to develop a scalable classifier system to solve high-dimensional problems. Recently work has begun on autonomously reusing learned building blocks of knowledge to scale from low-dimensional problems to high-dimensional ones. An XCS-based classifier system, known as XCSCFC, has been shown to be scalable, through the addition of expression tree-like code fragments, to a limit beyond standard learning classifier systems. XCSCFC is especially beneficial if the target problem can be divided into a hierarchy of subproblems and each of them is solvable in a bottom-up fashion. However, if the hierarchy of subproblems is too deep, then XCSCFC becomes impractical because of the needed computational time and thus eventually hits a limit in problem size. A limitation in this technique is the lack of a cyclic representation, which is inherent in finite state machines (FSMs). However, the evolution of FSMs is a hard task owing to the combinatorially large number of possible states, connections, and interaction. Usually this requires supervised learning to minimize inappropriate FSMs, which for high-dimensional problems necessitates subsampling or incremental testing. To avoid these constraints, this work introduces a state-machine-based encoding scheme into XCS for the first time, termed XCSSMA. The proposed system has been tested on six complex Boolean problem domains: multiplexer, majority-on, carry, even-parity, count ones, and digital design verification problems. The proposed approach outperforms XCSCFA (an XCS that computes actions) and XCSF (an XCS that computes predictions) in three of the six problem domains, while the performance in others is similar. In addition, XCSSMA evolved, for the first time, compact and human readable general classifiers (i.e., solving any n-bit problems) for the even-parity and carry problem domains, demonstrating its ability to produce scalable solutions using a cyclic representation.

Learning Classifier Systems (LCSs) originated from artificial cognitive systems, eventually they migrated such that LCS became powerful classification techniques. Current LCSs can extract building blocks of knowledge utilizing Code Fragments in order to scale to more difficult problems in the same or a related domain. Code Fragments (CF) are GP-like sub-trees where past learning can be reused in their leaf or root nodes. A downside is that the expressive alphabet used by the CFs requires more computing resources as the learned knowledge grows. The long chains of CFs that eventually appear make CFs incapable of scaling to more complex problems. Previous work shows that a new layer of Distilled Rules (DRs) created in batch mode after training, was beneficial in future reuse, but at the cost of long computational times. In the novel work here, an innovative online method to produce DRs is described and compared with the original method. The system has been tested on Boolean problems up to the 70 bit multiplexer and 3×11 bit hidden multiplexer, which are difficult problems for conventional algorithms. This is due to the large search spaces involved. The new technique has been shown to create a new layer of DRs for the 70 Mux, something that the previous version was unable to accomplish in a timely manner. It has also been able to scale to more difficult problems in the same or a related domain.

Figure-ground segmentation is an essential but difficult preprocessing step for many computer vision and image preprocessing tasks, such as object recognition. One challenge is to separate objects from backgrounds on images with high variations (e.g. in object shapes), which requires both effective feature sets and powerful segmentors. This paper develops a GP based segmentation method, which transforms segmentation tasks into pixel classification based problems. To control the complexity of evolved solutions, parsimony pressure is introduced in GP. Tested on two datasets with high variations (the Weizmann and Pascal datasets), the proposed method achieves similar performance in F\ score with much simpler solutions, compared with a reference GP based method that does not consider solution complexity. Moreover, it is the first time that the occurrence rates of the features used by the evolved solutions are studied to conduct feature selection for figure-ground segmentation. Compared with the whole feature set using traditional classifier based segmentation methods, the selected feature subsets can improve the segmentation performance. Moreover, analyses on the evolved solutions reveal how they function and why specific features are selected.

It has been shown that identifying building blocks of knowledge and then reusing them to solve complex problems is a practical and useful endeavor. Previous work made it possible to solve various, until then, intractable tasks. However, the individual algorithms targeted one specific problem type, e.g. scalable problems or domains with repeating patterns. The question that arises is: Can the disparate techniques be combined into a single approach to solve more complex problems that span several domains or that may be unknown to the agent? The first stage in developing such a system is to be able to recognise domains from unidentified input stimuli and identify the approaches best suited to them. The novel work here aims to realise this primary stage by combining several code-fragment (CF) based XCS systems. The stimulus and its guiding effect, will be instrumental in helping the agent decide which of its stored systems is the most capable of solving the problem, or if there is a conflict between possible solutions. Importantly, the agent will be capable of determining if the current problem is entirely new, in which case it spawns a training agent to produce a tractable solution to store and reuse. The proposed technique relies on the proven benefits in scalability of CF based systems and furthers the body of knowledge by tackling unknown problems (to the agent). The main contribution of this research is that a system of proven CF techniques is used for the first time. We show that by utilizing the new CF system, it is possible to identify an unknown problem and to arrive at a viable solution.

Genetic programming (GP) approaches have been widely studied for symbolic regression problems and have achieved substantial progress. This work investigates the effectiveness of niching property and multiple learned solutions of a Learning Classifier System (LCS) to symbolic regression benchmark problems. Specifically, an XCS with real-valued interval based conditions and code fragmented action termed as XCS-SR is proposed for tackling symbolic regression problem. This is the first LCS ever to address the problem of symbolic regression. The results on nine standard symbolic regression benchmarks show that the proposed XCS-SR method consistently obtains statistically better results on a majority of the benchmarks, in terms of average absolute error together with an increased number of exact solutions as compared with the GP benchmark.

Mammography is a well known procedure for breast cancer detection. The traditional mammography process employs manual analysis for detection and diagnosis, which requires professional expertise. However, computerized systems that use feature based classification have been demonstrated to be proficient and reliable but there is scope for improved accuracy. In this paper, we present the first comprehensive strategy to use learning classifier systems (LCSs) for mammogram image classification. We use six types of statistical measures, three different variants of local binary pattern (LBP) technique and ten variants of discrete wavelet transform (DWT) to extract statistical, texture and multiresolution features, respectively. However, the main challenge to apply an LCS in image classification tasks is the large number of extracted feature components that result in a large number of attributes in classifier conditions. Whereas, to evolve generalization in an LCS, a limited number of attributes in classifier conditions are required. We use different encoding schemes based on mapping distances against the large feature components to reduce the number of required attributes in classifier conditions while retaining the unique image characteristics. We develop a novel strategy that deploys various combinations of features and distances to investigate five different types of attributes in classifier conditions: (i) individual statistical features, (ii) individual LBP features, (iii) concatenation of statistical and LBP features, (iv) concatenation of statistical, LBP features and distances based on DWT features, and (v) concatenation of statistical features and distances based on LBP and DWT features. The obtained results indicate that using the precomputed distances in place of the original LBP and DWT features improve the classification accuracy in experiments conducted in this study.

Learning classifier systems (LCSs) originated from artificial cognitive systems research, but migrated such that LCS became powerful classification techniques. Modern LCSs can be used to extract building blocks of knowledge in order to solve more difficult problems in the same or a related domain. The past work showed that the reuse of knowledge through the adoption of code fragments, GP-like sub-trees, into the XCS learning classifier system framework could provide advances in scaling. However, unless the pattern underlying the complete domain can be described by the selected LCS representation of the problem, a limit of scaling will eventually be reached. This is due to LCSs' 'divide and conquer' approach utilizing rule-based solutions, which entails an increasing number of rules (subclauses) to describe a problem as it scales. Inspired by human problem solving abilities, the novel work in this paper seeks to reuse learned knowledge and learned functionality to scale to complex problems by transferring them from simpler problems. Progress is demonstrated on the benchmark Multiplexer (Mux) domain, albeit the developed approach is applicable to other scalable domains. The fundamental axioms necessary for learning are proposed. The methods for transfer learning in LCSs are developed. Also, learning is recast as a decomposition into a series of sub-problems. Results show that from a conventional tabula rasa, with only a vague notion of what subordinate problems might be relevant, it is possible to learn a general solution to any n-bit Mux problem for the first time. This is verified by tests on the 264, 521 and 1034 bit Mux problems.

Feature selection is an important task in data mining and machine learning to reduce the dimensionality of the data and increase the performance of an algorithm, such as a classification algorithm. However, feature selection is a challenging task due mainly to the large search space. A variety of methods have been applied to solve feature selection problems, where evolutionary computation (EC) techniques have recently gained much attention and shown some success. However, there are no comprehensive guidelines on the strengths and weaknesses of alternative approaches. This leads to a disjointed and fragmented field with ultimately lost opportunities for improving performance and successful applications. This paper presents a comprehensive survey of the state-of-the-art work on EC for feature selection, which identifies the contributions of these different algorithms. In addition, current issues and challenges are also discussed to identify promising areas for future research.

Salient object detection is typically accomplished by combining the outputs of multiple primitive feature detectors (that output feature maps or features). The diversity of images means that different basic features are useful in different contexts, which motivates the use of complementary feature detectors in a general setting. However, naive inclusion of features that are not useful for a particular image leads to a reduction in performance. In this paper, we introduce four novel measures of feature quality and then use those measures to dynamically select useful features for the combination process. The resulting saliency is thereby individually tailored to each image. Using benchmark data sets, we demonstrate the efficacy of our dynamic feature selection system by measuring the performance enhancement over the state-of-the-art models for complementary feature selection and saliency aggregation tasks. We show that a salient object detection technique using our approach outperforms competitive models on the PASCAL VOC 2012 dataset. We find that the most pronounced performance improvements occur in challenging images with cluttered backgrounds, or containing multiple salient objects.

A learning strategy in Learning Classifier Systems (LCSs) defines how classifiers cover a state-action space in a problem. Previous analyses in classification problems have empirically claimed an adequate learning strategy can be decided depending on the types of noise in the problem. This issue is still arguable from two aspects. First, there lacks comparison of learning strategies in reinforcement learning problems with different types of noise. Second, when we can claim so, a further issue is how should classifiers cover the state-action space in order to improve the stability of LCS performance on as many types of noise as possible? This paper first attempts to empirically conclude these issues on a version of LCSs (i.e., the XCS classifier system). That is, we present a new concept of learning strategy for LCSs, and complement that claim by comparing it with the existing learning strategies on a reinforcement learning problem. Our learning strategy covers all state-action pairs but assigns more classifiers to the highest-return action at each state than other actions. Our results support that claim that existing learning strategies have dependencies on the types of noise in reinforcement learning problems. However, our learning strategy improves the stability of XCS performance compared with the existing strategies on all types of noise employed in this paper.

This paper presents an emotion inspired adaptive path planning approach for autonomous robotic navigation. Ideally a robotic navigation system should adapt its path planning and behaviour to overcome a variety of obstacles within an environment, without the need for single location planning approaches. Emotional analogies are appealing as they enable general planning, but require hard coding of `emotions'. Humans have a bias on what is an emotion, e.g. fear, which can adversely affect performance. We aim to provide the robot with the generalising ability of emotion without the pre-specifying bias. Inspired by theories on `emotion', the system presented utilises a Learning Classifier System (LCS) to learn a `bow-tie' structure of emotional reinforcers to intermediary emotion categories to a behavioural modifier that adapts the robot's navigation behaviour. The emotional states are not pre-set and are judged post learning based on the learned behaviour. The bow-tie creates a simple compact set of rules to adapt a robot's behaviour to better navigate its environment. The emotion system was verified on a state-of-the-art navigation system to learn a variety of parameters that control the robot's behaviour. The results show two easy to understand learned emotional states; the first is considered to be a model `fear', which increases obstacle avoidance while lowering speed when pain is induced or novelty is high. The second emotion is considered to be `happiness', which increases speed and lowers wall avoidance when pain is not present. Compared to the default non-adapting navigation system, the emotional responses decreased the overall number of collisions and improved time to navigate.

Feature selection is an important data preprocessing step in machine learning and data mining, such as classification tasks. Research on feature selection has been extensively conducted for more than 50 years and different types of approaches have been proposed, which include wrapper approaches or filter approaches, and single objective approaches or multi-objective approaches. However, the advantages and disadvantages of such approaches have not been thoroughly investigated. This paper provides a comprehensive study on comparing different types of feature selection approaches, specifically including comparisons on the classification performance and computational time of wrappers and filters, generality of wrapper approaches, and comparisons on single objective and multi-objective approaches. Particle swarm optimization (PSO)-based approaches, which include different types of methods, are used as typical examples to conduct this research. A total of 10 different feature selection methods and over 7000 experiments are involved. The results show that filters are usually faster than wrappers, but wrappers using a simple classification algorithm can be faster than filters. Wrappers often achieve better classification performance than filters. Feature subsets obtained from wrappers can be general to other classification algorithms. Meanwhile, multi-objective approaches are generally better choices than single objective algorithms. The findings are not only useful for researchers to develop new approaches to addressing new challenges in feature selection, but also useful for real-world decision makers to choose a specific feature selection method according to their own requirements.

Combining features from different modalities and domains has been demonstrated to enhance the performance of saliency prediction algorithms. Different feature combinations are often suited to different types of images, but existing techniques attempt to apply a single feature combination across all image types. Furthermore, existing normalization and integration schemes are not utilized in salient object detection as the combination of potential solutions is intractable to test. The aim of this work is to autonomously learn feature combinations for autonomously identified image types. To this end, we learn optimal normalization and integration schemes along with feature weightings using a novel Genetic Algorithm (GA) method. Moreover, we learn multiple image dependent parameters using our novel image-based GA (IGA) approach, to increase the generalization of the system on unseen test images. We present a thorough quantitative and qualitative comparison of our proposed methods with the state-of-the-art benchmark and deterministic methods on two difficult datasets (SED1 and SED2) from the segmentation evaluation database. IGA shows superior performance through learning optimal parameters depending upon the composition of images and using feature combinations appropriately enhances test performance and generalization of the system.

We established an engineering degree utilising existing science and mathematics courses where possible in order to minimise the resource requirements. After 7 years of running this degree, research indicated dissatisfaction by some students regarding the science, and in particular, mathematics component of the programme. We also uncovered numerous non-academic issues that contributed to student disengagement from the degree. This paper outlines the evolution of an engineering degree from its inception to its current form. This evolution is informed by student surveys, focus groups, interviews, and best practice. The result has been a significant change to the foundation engineering course, the creation of new engineering courses, the appointment of a pastoral support agent, the growing of a student engineering culture and the redevelopment and re-emphasising of first year mathematics.

Learning classifier systems (LCSs) are rule-based online evolutionary machine learning techniques that solve a problem by interacting with an environment. LCSs have been successfully used in various applications such as data mining, robot control and computer vision systems. Salient object detection is the task of automatically localizing the objects of interests in a scene by suppressing the background information, which facilitates various machine learning applications such as object segmentation, recognition and tracking. It is a difficult problem as natural scenes can often have objects with cluttered backgrounds (making it difficult to distinguish the object from background based on its features) or other complicating factors such as multiple objects. Existing saliency learning methods learn a single weight vector emphasizing the importance of each feature/attribute for the whole image dataset, hence losing generalization in the test phase when considering unseen images. LCS technique has the ability to learn weight sets for different types of images automatically. Hence, this paper investigates the application of LCS for learning image dependent feature fusion strategies for the task of salient object detection. Our LCS approach evolves generalized rules for a well known benchmark dataset consisting of 1000 images, of various types and difficulty levels, and outperforms a genetic algorithm based system that was previously state-of-the-art.

The Three-Cornered Coevolution concept describes a framework where artificial problems may be generated in concert with classification agents in order to provide insight into their relationships. This is unlike standard studies where humans set a problem's difficulty, which may have bias or lack understanding of the multiple interactions of a problem's characteristics, such as noise in conjunction with class imbalance. Previous studies have shown that it is feasible to generate problems with one agent in relation to a single classification agent's performance, but when to adjust the problem difficulty was manually set. This paper introduces a second classification agent to trigger the coevolutionary process within the system, where its functionality and effect on the system requires investigation. The classification agents, in this case Learning Classifier Systems, use different styles of learning techniques (e.g. supervised or reinforcement learning techniques) to learn the problems. Experiments show that the realized system is capable of autonomously generating various problems, triggering learning and providing insight into each learning system's ability by determining the problem domains where they perform relatively well - this is in contrast to humans having to determine the problem domains.

Image pattern classification is a challenging task due to the large search space of pixel data. Supervised and subsymbolic approaches have proven accurate in learning a problem's classes. However, in the complex image recognition domain, there is a need for investigation of learning techniques that allow humans to interpret the learned rules in order to gain an insight about the problem. Learning classifier systems (LCSs) are a machine learning technique that have been minimally explored for image classification. This work has developed the feature pattern classification system (FPCS) framework by adopting Haar-like features from the image recognition domain for feature extraction. The FPCS integrates Haar-like features with XCS, which is an accuracy-based LCS. A major contribution of this work is that the developed framework is capable of producing human-interpretable rules. The FPCS system achieved 91 [Formula: see text] 1% accuracy on the unseen test set of the MNIST dataset. In addition, the FPCS is capable of autonomously adjusting the rotation angle in unaligned images. This rotation adjustment raised the accuracy of FPCS to 95%. Although the performance is competitive with equivalent approaches, this was not as accurate as subsymbolic approaches on this dataset. However, the benefit of the interpretability of rules produced by FPCS enabled us to identify the distribution of the learned angles-a normal distribution around [Formula: see text]-which would have been very difficult in subsymbolic approaches. The analyzable nature of FPCS is anticipated to be beneficial in domains such as speed sign recognition, where underlying reasoning and confidence of recognition needs to be human interpretable.

Feature selection is a multi-objective problem, where the two main objectives are to maximize the classification accuracy and minimize the number of features. However, most of the existing algorithms belong to single objective, wrapper approaches. In this work, we investigate the use of binary particle swarm optimization (BPSO) and probabilistic rough set (PRS) for multi-objective feature selection. We use PRS to propose a new measure for the number of features based on which a new filter based single objective algorithm (PSOPRSE) is developed. Then a new filter-based multi-objective algorithm (MORSE) is proposed, which aims to maximize a measure for the classification performance and minimize the new measure for the number of features. MORSE is examined and compared with PSOPRSE, two existing PSO-based single objective algorithms, two traditional methods, and the only existing BPSO and PRS-based multi-objective algorithm (MORSN). Experiments have been conducted on six commonly used discrete datasets with a relative small number of features and six continuous datasets with a large number of features. The classification performance of the selected feature subsets are evaluated by three classification algorithms (decision trees, Naïve Bayes, and k-nearest neighbors). The results show that the proposed algorithms can automatically select a smaller number of features and achieve similar or better classification performance than using all features. PSOPRSE achieves better performance than the other two PSO-based single objective algorithms and the two traditional methods. MORSN and MORSE outperform all these five single objective algorithms in terms of both the classification performance and the number of features. MORSE achieves better classification performance than MORSN. These filter algorithms are general to the three different classification algorithms.

Evolutionary computation techniques have had limited capabilities in solving large-scale problems due to the large search space demanding large memory and much longer training times. In the work presented here, a genetic programming like rich encoding scheme has been constructed to identify building blocks of knowledge in a learning classifier system. The fitter building blocks from the learning system trained against smaller problems have been utilized in a higher complexity problem in the domain to achieve scalable learning. The proposed system has been examined and evaluated on four different Boolean problem domains: 1) multiplexer, 2) majority-on, 3) carry, and 4) even-parity problems. The major contribution of this paper is to successfully extract useful building blocks from smaller problems and reuse them to learn more complex large-scale problems in the domain, e.g., 135-bit multiplexer problem, where the number of possible instances is 2135≈ 4 × 1040, is solved by reusing the extracted knowledge from the learned lower level solutions in the domain. Autonomous scaling is, for the first time, shown to be possible in learning classifier systems. It improves effectiveness and reduces the number of training instances required in large problems, but requires more time due to its sequential build-up of knowledge.

XCS is an accuracy-based learning classifier system, which has been successfully applied to learn various classification and function approximation problems. Recently, it has been reported that XCS cannot learn overlapping natured and niche imbalance problems using the typical experimental setup. Previously we have developed an XCS with code-fragment action, named XCSCFA, which has the unusual property that during training the action value in a classifier rule can vary, even for the same problem instance, at different times. In the work presented here, the XCSCFA approach is applied to four different complex Boolean problem domains including the overlapping natured and niche imbalance domains. The XCSCFA system successfully learnt all the experimented problems. The major contribution of this work is overcoming the identified problem in the widespread XCS technique, i.e. it is no longer impossible to learn overlapping natured and niche imbalance problems.

Genetic programming for evolutionary art often focuses on improving fitness functions to improve image quality. We take the opposite approach, to determine how influential the internal representation is on the expression of images. We define four primitive sets based on common ideas from the literature and compare the resulting images based on a constant fitness function. Although it is obvious that changing the primitive set has an effect on the resulting images and their fitness, it has not been thoroughly investigated. This paper explores the effect of changing primitive sets in genetic programming for evolutionary art. We find that different primitive sets have different effects on how the final image looks as well as how it is affected by genetic operators. We find that geometric primitive sets are better for creating recognisable images, and are able to make small localised changes to the image over generations. In contrast the mathematical primitive sets result in intricate patterns over the whole image, and a small change in the tree can result in a change across the whole image rather than in a localised area.

Predicting where humans look in a scene is crucial in tasks like human-computer interaction, design, graphics, image and video compression, and gaze animation. This work proposes the use of a mixed-integer constraint Genetic Algorithm (GA) for searching the optimal parameters of a bio-inspired visual saliency model for accurate prediction of human eye fixations. Bio-inspired visual saliency models are complex models, mimicking the primate visual system with a vast choice of design parameters that can be tuned to achieve optimal performance. The bottomup visual attention model used in this study was trained on three challenging image datasets from the ImgSal database using a standard performance metric (area under Receiver Operating Characteristic curve) as the fitness. To compensate for any bias of the optimized model towards the standard metric, we use two other scoring metrics to assess performance. Performance comparisons with eight state-of-the-art models have been presented for all three scoring metrics. Results show that the proposed GA optimized visual attention model provides better prediction performance than several state-of-the-art models of visual attention.

The assessment of sites for liquefaction potential in earthquakes currently relies on the estimation of soil layer models which is laborious and standard regression techniques ineffectual. Although Parallel Linear Genetic Programming (PLGP) has proven to be an effective method for classification tasks it has not yet been applied to regression problems. This paper redefines a time-consuming, operator intensive process as an Evolutionary Computation (EC) regression task and designs a PLGP system that can produce candidate solutions for an operator to review. This paper introduces Evolutionary Spatial Auto-Correlation (ESPAC) which is an EC technique that uses a similar structure to PLGP programs to represent some layer models and evolve them using error matching against the target curve as a fitness function. The project achieves its goal of providing a working proof-of-concept with resultant curve matching being improved over that of a domain expert on four of the five datasets tested.

Evolutionary computational techniques have had limited capabilities in solving large-scale problems, due to the large search space demanding large memory and much longer training time. Recently work has begun on automously reusing learnt building blocks of knowledge to scale from low dimensional problems to large-scale ones. An XCS-based classifier system has been shown to be scalable, through the addition of tree-like code fragments, to a limit beyond standard learning classifier systems. Self-modifying cartesian genetic programming (SMCGP) can provide general solutions to a number of problems, but the obtained solutions for large-scale problems are not easily interpretable. A limitation in both techniques is the lack of a cyclic representation, which is inherent in finite state machines. Hence this work introduces a state-machine based encoding scheme into scalable XCS, for the first time, in an attempt to develop a general scalable classifier system producing easily interpretable classifier rules. The proposed system has been tested on four different Boolean problem domains, i.e. even-parity, majority-on, carry, and multiplexer problems. The proposed approach outperformed standard XCS in three of the four problem domains. In addition, the evolved machines provide general solutions to the even-parity and carry problems that are easily interpretable as compared with the solutions obtained using SMCGP.

In classification, the quality of the data representation significantly influences the performance of a classification algorithm. Feature construction can improve the data representation by constructing new high-level features. Particle swarm optimisation (PSO) is a powerful search technique, but has never been applied to feature construction. This paper proposes a PSO based feature construction approach (PSOFC) to constructing a single new high-level feature using original low-level features and directly addressing binary classification problems without using any classification algorithm. Experiments have been conducted on seven datasets of varying difficulty. Three classification algorithms (decision trees, naive bayes, and k-nearest neighbours) are used to evaluate the performance of the constructed feature on test set. Experimental results show that a classification algorithm using the single constructed feature often achieves similar (or even better) classification performance than using all the original features, and in almost all cases, adding the constructed feature to the original features significantly improves its classification performance. In most cases, PSOFC as a classification algorithm (using the constructed feature only) achieves better classification performance than a classification algorithm using all the original features, but needs much less computational cost. This paper represents the first study on using PSO for feature construction in classification.

Classification problems often have a large number of features in the data sets, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the performance. Feature selection aims to choose a small number of relevant features to achieve similar or even better classification performance than using all features. It has two main conflicting objectives of maximizing the classification performance and minimizing the number of features. However, most existing feature selection algorithms treat the task as a single objective problem. This paper presents the first study on multi-objective particle swarm optimization (PSO) for feature selection. The task is to generate a Pareto front of nondominated solutions (feature subsets). We investigate two PSO-based multi-objective feature selection algorithms. The first algorithm introduces the idea of nondominated sorting into PSO to address feature selection problems. The second algorithm applies the ideas of crowding, mutation, and dominance to PSO to search for the Pareto front solutions. The two multi-objective algorithms are compared with two conventional feature selection methods, a single objective feature selection method, a two-stage feature selection algorithm, and three well-known evolutionary multi-objective algorithms on 12 benchmark data sets. The experimental results show that the two PSO-based multi-objective algorithms can automatically evolve a set of nondominated solutions. The first algorithm outperforms the two conventional methods, the single objective method, and the two-stage algorithm. It achieves comparable results with the existing three well-known multi-objective algorithms in most cases. The second algorithm achieves better results than the first algorithm and all other methods mentioned previously.

A cognitive mobile robot must be able to autonomously solve the three complex problems of navigating: where it is, where it is going and how it is going to get there. The first is addressed by techniques for simultaneous localization and mapping (SLAM). The next stage of navigating is to plan a path to a goal, which is often achieved by learning techniques due to the scale of search required. Commonly, the localisation and mapping stage is separated from path planning stage, with the function not of interest being considered ideal in order to simplify the problem (similarly, the goal is often predetermined by an external agent, such as a human operator specifying a location to reach). This work integrates the planning with the localisation and mapping in order to investigate the benefits of considering these aspects together (rather than as a separate functions as is often assumed). Firstly, experiments on real-robots show decreased localisation error in this approach (1.8 mm ±0.41 mm to 1.2 mm ±0.26 mm). Secondly, the number of steps to goal has concurrently been reduced (13.4 steps to 11.8 steps). This work is novel in the integration of evolutionary computation planning techniques with SLAM. It also has enabled the opportunity for rule-sharing between heterogeneous robots and the inclusion of action policies in SLAM filter updates.

Feature selection is an important data preprocessing technique in classification problems. This paper proposes two new fitness functions in binary particle swarm optimisation (BPSO) for feature selection to choose a small number of features and achieve high classification accuracy. In the first fitness function, the relative importance of classification performance and the number of features are balanced by using a linearly increasing weight in the evolutionary process. The second is a two-stage fitness function, where classification performance is optimised in the first stage and the number of features is taken into account in the second stage. K-nearest neighbour (KNN) is employed to evaluate the classification performance in the experiments on ten datasets. Experimental results show that by using either of the two proposed fitness functions in the training process, in almost all cases, BPSO can select a smaller number of features and achieve higher classification accuracy on the test sets than using overall classification performance as the fitness function. They outperform two conventional feature selection methods in almost all cases. In most cases, BPSO with the second fitness function can achieve better performance than with the first fitness function in terms of classification accuracy and the number of features.

The New Zealand Government is moving towards restricting access to tertiary education and implementing a managed entry scheme. It is therefore important to be able to predict whether a student has a reasonable likelihood of succeeding in tertiary engineering study. New Zealand secondary schools mostly operate on the National Certificate of Educational Achievement (NCEA) model whereby subjects are assessed on the basis of discrete individual modules. The paper compares the NCEA results to first year grades in tertiary engineering subjects obtained from most New Zealand providers of the BE degree to determine whether these NCEA grades can be used as a predictor of success or failure in tertiary engineering programmes. This is the first nation-wide survey of its kind and has yielded surprising results. For example, predicting success based on whether a student has not achieved is more insightful than basing on achievements, which is counter to the basis of many school league tables worldwide.

Victoria University of Wellington in partnership with the regional polytechnic, WelTec, undertook a major exercise to identify, and where possible, resolve, barriers to recruitment and retention in the “digital” engineering specializations. This paper focuses on the retention aspects of this research. Informed by student surveys, focus groups and secondary school academic achievement data, we identified contributing issues of academic preparation, student expectation and cultural influencers. In response we developed an engineering preparation course, a mathematics based diagnostic tool, Peer-Assisted learning support, engineering cultural activities, and redeveloped our core first year engineering course. Although in the early stages of delivery, these initiatives have been well received by the students. We are closely monitoring the results of these initiatives with the expectation that fewer students will abandon their studies and a greater portion of the marginal students will attain passing grades.

Extracting features from images is an important task in order to identify (classify) the patterns contained. The Evolutionary Computation and Reinforcement Learning technique of Learning Classifier Systems (LCSs) has been successfully applied to classification tasks, but rarely to image pattern classification due to the large search space associated with pixel data. Recently, a Feature Pattern Classification System (FPCS), utilising Haar-like features has been introduced with promising results in the image recognition domain. This system used a confusion-matrix to direct learning to hard to classify classes, but due to its reinforcement learning nature was required to estimate the ground truth. The novel work presented here adopts a supervised learning (UCS-based) approach into the FPCS framework. This work is compared with the original XCS-based system, updated to include the known ground-truth of the confusion matrix to aid comparison, albeit no longer reinforcement learning. Results on the 10 class MNIST numerical digits recognition task show that the XCS-based FPCS produces better classification due to its complete mapping guiding learning. However, results on the 26 class NIST character recognition task show that the UCS-based scales better as it does not require the complete mapping. The human readable rules produced by each system, coupled with the competitive classification performance compared with similar techniques, supports future work on both the XCS and UCS-based FPCS.

Human beings have the ability to apply the domain knowledge learned from a smaller problem to more complex problems of the same or a related domain, but currently evolutionary computation techniques lack this ability. Hence these techniques relearn from the start when the problem scales, increasing the time required and potentially limiting capability. In order to autonomously scale in a problem domain reusable building blocks of knowledge must be extracted. A richer encoding scheme than ternary alphabet has been constructed to identify building blocks. The novel work presented here is to extract useful building blocks from smaller problems and reuse them to learn complex problems in the domain. The proposed system has been compared with ternary alphabet based XCS for three different problem domains, i.e. multiplexer, carry, and even-parity problems. Autonomous scaling is shown possible for the first time in learning classifier systems. It improves effectiveness and reduces the number of training instances required in large problems, but requires more time due to more involved methods.

Classifying objects and patterns to a certain category is crucial for both humans and machines, so that learnt knowledge may be applied across similar problem instances. Although autonomous learning of patterns by machines has advanced recently, it still requires humans to set up the problem at an appropriate level for the learning technique. If the problem is too complex the system does not learn; conversely, if the problem is too simple the system does not reach its full potential to be able to classify environmental examples. In this work, an automated evolving pattern generator and pattern recognizer has been created for pattern classification problems that can be manipulated autonomously using Learning Classifier Systems (LCSs) at different levels of difficulty. Experiments confirm that both of the agents (e.g. the pattern generation and the pattern classification agent) can be evolved autonomously and co-operatively. The novel contributions in this work enable the effect of domain features on classification performance to become human readable, i.e. possibly determine what features make it difficult for the classification algorithm to learn. This work provides a foundation for a co-evolutionary approach to problem domain creation and the associated learning, such that the agents will trigger evolution when necessary.

Feature selection (FS) is an important data preprocessing technique, which has two goals of minimising the classification error and minimising the number of features selected. Based on particle swarm optimisation (PSO), this paper proposes two multi-objective algorithms for selecting the Pareto front of non-dominated solutions (feature subsets) for classification. The first algorithm introduces the idea of non-dominated sorting based multi-objective genetic algorithm II into PSO for FS. In the second algorithm, multi-objective PSO uses the ideas of crowding, mutation and dominance to search for the Pareto front solutions. The two algorithms are compared with two single objective FS methods and a conventional FS method on nine datasets. Experimental results show that both proposed algorithms can automatically evolve a smaller number of features and achieve better classification performance than using all features and feature subsets obtained from the two single objective methods and the conventional method. Both the continuous and the binary versions of PSO are investigated in the two proposed algorithms and the results show that continuous version generally achieves better performance than the binary version. The second new algorithm outperforms the first algorithm in both continuous and binary versions.

Feature selection has the two main objectives of minimising the classification error rate and the number of features. Based on binary particle swarm optimisation (BPSO), we develop two novel multi-objective feature selection frameworks for classification, which are multi-objective binary PSO using the idea of non-dominated sorting (NSBPSO) and multi-objective binary PSO using the ideas of crowding, mutation and dominance (CMDBPSO). Four multi-objective feature selection methods are then developed by applying mutual information and entropy as two different filter evaluation criteria in each of the proposed frameworks. The proposed algorithms are examined and compared with a single objective method on eight benchmark data sets. Experimental results show that the proposed multi-objective algorithms can evolve a set of solutions that use a smaller number of features and achieve better classification performance than using all features. In most cases, NSBPSO achieves better results than the single objective algorithm and CMDBPSO outperforms all other methods mentioned above. This work represents the first study on multi-objective BPSO for filter-based feature selection.

Evolutionary Art is a sub-field of Evolutionary Computing that involves creating interesting images using Evolutionary Techniques. Previously Genetic Programming has been used to create such images autonomously - that is, without a human in the loop. However, this work did not explore alternative fitness measures, consider colour in fitness or provide independent validation of results. Four fitness functions based on the concept that the pleasingness of an image is based on the ratio of image complexity to processing complexity are explored. We introduce the use of Shannon Entropy as a measure of image complexity to compare with Jpeg Compression. Similarly, we introduce Run Length Encoding to compare with Fractal Compression as a measure of processing complexity. A survey of 100 participants showed that it is possible to generate aesthetically pleasing graphics using each fitness function. Importantly, it was the introduction of colour that separated the aesthetic effects of the fitness measures.

Loops are an important part of classic programming techniques, but are rarely used in genetic programming. This paper presents a method of using unrestricted, i.e. nesting, loops to evolve programs for image classification tasks. Contrary to many other classification methods where pre-extracted features are typically used, we perform calculations on image regions determined by the loops. Since the loops can be nested, these regions may depend on previously computed regions, thereby allowing a simple version of conditional evaluation. The proposed GP approach with unrestricted loops is examined and compared with the canonical GP method without loops and the GP approach with restricted loops on one synthesized character recognition problem and two texture classification problems. The results suggest that unrestricted loops can have an advantage over the other two methods in certain situations for image classification.

Genetic programming (GP) has been successfully applied to solving multiclass classification problems, but the performance of GP classifiers still lags behind that of alternative techniques. This paper investigates an alternative form of GP, Linear GP (LGP), which demonstrates great promise as a classifier as the division of classes is inherent in this technique. By combining biological inspiration with detailed knowledge of program structure two new crossover operators that significantly improve performance are developed. The first is a new crossover operator that mimics biological crossover between alleles, which helps reduce the disruptive effect on building blocks of information. The second is an extension of the first where a heuristic is used to predict offspring fitness guiding search to promising solutions.

Outlier detection is an important problem as the underlying data points often contain crucial information, but identifying such points has multiple difficulties, e.g. noisy data, imprecise boundaries and lack of training examples. In this novel approach, the outlier detection problem is converted into an optimisation problem. A Particle Swarm Optimisation (PSO) based approach to outlier detection can then be applied, which expands the scope of PSO and enables new insights into outlier detection. Namely, PSO is used to automatically optimise the key distance measures instead of manually setting the distance parameters via trial and error, which is inefficient and often ineffective. The novel PSO approach is examined and compared with a commonly used detection method, Local Outlier Factor (LOF), on five real data sets. The results show that the new PSO method significantly outperforms the LOF methods for correctly detecting the outliers on the majority of the datasets and that the new PSO method is more efficient than the LOF method on the datasets tested.

The study of multi-criterion minimum spanning trees is important as many optimization problems in networks, such as communication, transport and utilities can be represented by this model. Conventional evolutionary approaches struggle to discover near-optimal solutions due to the combinatorial search space, and the difficulty in discovering the non-supported solutions. Recently, a knowledge-based evolutionary approach, KEA, has been developed that overcomes some of the problems of the earlier algorithms as it is not restricted to the bi-criterion case, finds non-supported solutions and scales well to larger problems; however, the mid-point of its Pareto front is often dominated by alternative algorithms where they are applicable. Novel extensions to KEA, increasing the knowledge of the mid-point, termed KEA-W are examined, eliminating the mid-point deficiencies at the cost of computational time.

A hybridised and Knowledge-based Evolutionary Algorithm (KEA) is applied to the multi-criterion minimum spanning tree problems. Hybridisation is used across its three phases. In the first phase a deterministic single objective optimization algorithm finds the extreme points of the Pareto front. In the second phase a K-best approach finds the first neighbours of the extreme points, which serve as an elitist parent population to an evolutionary algorithm in the third phase. A knowledge-based mutation operator is applied in each generation to reproduce individuals that are at least as good as the unique parent. The advantages of KEA over previous algorithms include its speed (making it applicable to large real-world problems), its scalability to more than two criteria, and its ability to find both the supported and unsupported optimal solutions.

A fast Knowledge-based Evolution Strategy, KES, for the multi-objective minimum spanning tree, is presented. The proposed algorithm is validated, for the bi-objective case, with an exhaustive search for small problems (4-10 nodes), and compared with a deterministic algorithm, EPDA and NSGA-II for larger problems (up to 100 nodes) using benchmark hard instances. Experimental results show that KES finds the true Pareto fronts for small instances of the problem and calculates good approximation Pareto sets for larger instances tested. It is shown that the fronts calculated by KES are superior to NSGA-II fronts and almost as good as those established by EPDA. KES is designed to be scalable to multi-objective problems and fast due to its small complexity.

Higher order networks allow modelling of correlates and geometrically invariant properties. Current techniques for their development either require domain knowledge, or are constrained by scaling properties or local minima. A novel reformulation of the product unit is introduced, motivated by a desire to improve scaling and training properties. The new unit allows developing high orders of positive and negative powers, and correlates in a single stage, but can be trained successfully using standard back propagation techniques. Tests on standard benchmarks in various hybrid topologies demonstrate the potential in a variety of problem domains.

Abstraction is a higher order cognitive ability that facilitates the production of rules that are independent of their associations. Experience from real-world data-mining has shown the need for such higher level rules. The game of Connect 4 is both multistep and complex, so standard Q-learning and Learning Classifier Systems perform poorly. The introduction of a novel Abstraction algorithm into an LCS is shown to improve performance in the evolution of playing strategies.