Logical reasoning is a major attribute of human intelligence. Humans demonstrate remarkable proficiency in combining information from multiple modalities for logical reasoning. This capability mirrors the tasks performed by Visual Question Answering (VQA), which is a complex task that requires an understanding of both visual elements and language. However, existing methods, which often rely on deep neural networks to learn implicit representations from data, lack the capacity to reason complex logical problems and provide interpretable explanations. To address this challenge, we propose a novel approach that combines hybrid neural-symbolic reasoning and explainable AI to enhance the explicit reasoning capabilities of VQA models when addressing complex logic questions. Our approach comprises two main components: the Neural-Logical Reasoning Network (NLRN) with the Comprehensive Logical Composition Attention Mechanism (CLoCA), which trains and applies predefined logical rules within the neural network to effectively reason about complex logical questions in VQA tasks; and an explainable AI module that uses symbolic AI methods to provide explanations of the neural network’s decision-making process in VQA. We evaluate our methodology on LoRA, a complex logical reasoning VQA dataset and other recent VQA datasets, demonstrating state-of-the-art performance while providing interpretable and accurate explanations.

State-of-the-art Visual Question Answering (VQA) systems have demonstrated promising performance in solving visual relationship-based reasoning problems. However, they struggle in solving complex problems where the answers require sophisticated logical reasoning. In this paper, we introduce a hybrid symbolic-neural reasoning model that integrates deep neural network vision and language features with a symbolic reasoner connected to a knowledge base. The symbolic reasoner effectively combines visual and linguistic information with on-tological relationships and common-sense reasoning to address complex logical questions. We replace the multimodal fusion layer in traditional VQA deep neural networks with an innovative logical reasoning component, generating reasoned answers and clear logical inference chains. Moreover, we propose developing a notion of Question Difficulty, reflecting the logical complexity of VQA questions and their difficulty level in terms of being answered. Current VQA approaches excel at straightforward logic but struggle with increased question difficulty. Our hybrid method performs better as it has access to an additional logical reasoner through the knowledge base to produce answers that require logical inference. Experimental analysis of the answers and the key evidential predicates generated using our unique LoRA (Logical Reasoning Associated VQA) dataset are used to validate our approach and clearly demonstrate its advantages.

The capacity to reason logically is a hallmark of human cognition. Humans excel at integrating multimodal information for locigal reasoning, as exemplified by the Visual Question Answering (VQA) task, which is a challenging multimodal task. VQA tasks and large vision-and-language models aim to tackle reasoning problems, but the accuracy, consistency and fabrication of the generated answers is hard to evaluate in the absence of a VQA dataset that can offer formal, comprehensive and systematic complex logical reasoning questions. To address this gap, we present LoRA, a novel Logical Reasoning Augmented VQA dataset that requires formal and complex description logic reasoning based on a food-and-kitchen knowledge base. Our main objective in creating LoRA is to enhance the complex and formal logical reasoning capabilities of VQA models, which are not adequately measured by existing VQA datasets. We devise strong and flexible programs to automatically generate 200,000 diverse description logic reasoning questions based on the SROIQ Description Logic, along with realistic kitchen scenes and ground truth answers. We fine-tune the latest transformer VQA models and evaluate the zero-shot performance of the state-of-the-art large vision-and-language models on LoRA. The results reveal that LoRA presents a unique challenge in logical reasoning, setting a systematic and comprehensive evaluation standard.

We present Nonparametric Approximation of Inter-Trace returns (NAIT), a Reinforcement Learning algorithm for discrete action, pixel-based environments that is both highly sample and computation efficient. NAIT is a lazy-learning approach with an update that is equivalent to episodic Monte-Carlo on episode completion, but that allows the stable incorporation of rewards while an episode is ongoing. We make use of a fixed domain-agnostic representation, simple distance based exploration and a proximity graph-based lookup to facilitate extremely fast execution. We empirically evaluate NAIT on both the 26 and 57 game variants of ATARI100k where, despite its simplicity, it achieves competitive performance in the online setting with greater than 100x speedup in wall-time.

Dialogue Act tagging with the ISO 24617-2 standard is a difficult task that involves multi-label text classification across a diverse set of labels covering semantic, syntactic and pragmatic aspects of dialogue. The lack of an adequately sized training set annotated with this standard is a major problem when using the standard in practice. In this work we propose a neural architecture to increase classification accuracy, especially on low-frequency fine-grained tags. Our model takes advantage of the hierarchical structure of the ISO taxonomy and utilises syntactic information in the form of Part-Of-Speech and dependency tags, in addition to contextual information from previous turns. We train our architecture on an aggregated corpus of conversations from different domains, which provides a variety of dialogue interactions and linguistic registers. Our approach achieves state-of-the-art tagging results on the DialogBank benchmark data set, providing empirical evidence that this architecture can successfully generalise to different domains.

We introduce Retrieval Augmented Classification (RAC), a generic approach to augmenting standard image classification pipelines with an explicit retrieval module. RAC consists of a standard base image encoder fused with a parallel retrieval branch that queries a non-parametric external memory of pre-encoded images and associated text snippets. We apply RAC to the problem of long-tail classification and demonstrate a significant improvement over previous state-of-the-art on Places365-LT and iNaturalist-2018 (14.5% and 6.7% respectively), despite using only the training datasets themselves as the external information source. We demonstrate that RAC's retrieval module, without prompting, learns a high level of accuracy on tail classes. This, in turn, frees the base encoder to focus on common classes, and improve its performance thereon. RAC represents an alternative approach to utilizing large, pretrained models without requiring fine-tuning, as well as a first step towards more effectively making use of external memory within common computer vision architectures.

Vertebral compression fractures (VCF) often miss detection in radiology scans, risking more severe secondary fractures in the future leading to permanent disability and death. Automated solutions are therefore desirable, however a frequent bottleneck in medical image analysis is the availability of radiologist’s time for annotations. To alleviate this problem, this work presents the first attempt at VCF detection using Multiple Instance Learning (MIL), a weakly supervised learning approach that can cope with a small annotated data set. The method involves localisation of the thoracic and lumbar spine regions by generating 6 bounding boxes from which 2D patches are extracted. These patches are then used as instances in a bag within an MIL setting to train a deep learning architecture using an algorithm employing an embedded space paradigm with a shared convolutional neural network (CNN) layer. Majority voting is then performed on the results of the 6 bounding boxes to achieve accuracy / F1 score of 81.05% / 80.74% for thoracic and 85.45 % / 85.61% for lumbar spine respectively.

Several regularization methods have recently been introduced which force the latent activations of an autoencoder or deep neural network to conform to either a Gaussian or hyperspherical distribution, or to minimize the implicit rank of the distribution in latent space. In the present work, we introduce a simple and novel regularizing loss function which simulates a pairwise repulsive force between items and an attractive force of each item toward the origin. We show that minimizing this loss function in isolation achieves a hyperspherical distribution, and demonstrate its effectiveness as a regularizer for an image auto-encoder. Moreover, a reduction in the regularization parameter leads to a modest increase in the eccentricity of the distribution in latent space. This enhances image generation, and allows the eigenvectors of the covariance matrix to be extracted as deep principal components, which can be used for data analysis, image generation, visualization and downstream classification.

In this paper, we propose a feature-free method for detecting phishing websites using the Normalized Compression Distance (NCD), a parameter-free similarity measure which computes the similarity of two websites by compressing them, thus eliminating the need to perform any feature extraction. It also removes any dependence on a specific set of website features. This method examines the HTML of webpages and computes their similarity with known phishing websites, in order to classify them. We use the Furthest Point First algorithm to perform phishing prototype extractions, in order to select instances that are representative of a cluster of phishing webpages. We also introduce the use of an incremental learning algorithm as a framework for continuous and adaptive detection without extracting new features when concept drift occurs. On a large dataset, our proposed method significantly outperforms previous methods in detecting phishing websites, with an AUC score of 98.68%, a high true positive rate (TPR) of around 90%, while maintaining a low false positive rate (FPR) of 0.58%. Our approach uses prototypes, eliminating the need to retain long term data in the future, and is feasible to deploy in real systems with a processing time of roughly 0.3 seconds.

In this study, we build upon a previously proposed neuroevolution framework to evolve deep convolutional models. Specifically, the genome encoding and the crossover operator are extended to make them applicable to layered networks. We also propose a convolutional layer layout which allows kernels of different shapes and sizes to coexist within the same layer, and present an argument as to why this may be beneficial. The proposed layout enables the size and shape of individual kernels within a convolutional layer to be evolved with a corresponding new mutation operator. The proposed framework employs a hybrid optimisation strategy involving structural changes through epigenetic evolution and weight update through backpropagation in a population-based setting. Experiments on several image classification benchmarks demonstrate that the crossover operator is sufficiently robust to produce increasingly performant offspring even when the parents are trained on only a small random subset of the training dataset in each epoch, thus providing direct confirmation that learned features and behaviour can be successfully transferred from parent networks to offspring in the next generation.

There has recently been renewed interest in the paradigm of artist-critic coevolution, or adversarial training, in which an artist tries to generate images which are similar in style to a set of real images, and a critic tries to discriminate between the real images and those generated by the artist. We explore a novel configuration of this paradigm, where the artist is trained by hierarchical evolution using an evolutionary automatic programming language called HERCL, and the critic is a convolutional neural network. The system implicitly solves the constrained optimization problem of generating images which have low algorithmic complexity, but are sufficiently suggestive of real-world images as to fool a trained critic with an architecture loosely modeled on the human visual system. The resulting images are not necessarily photorealistic, but often consist of geometric shapes and patterns which remind us of everyday objects, landscapes or designs in a manner reminiscent of abstract art. We explore the coevolutionary dynamics between artist and critic, and discuss possible combinations of this framework with interactive evolution or other human-in-the-loop paradigms.

This paper introduces a speciation principle for neuroevolution where evolving networks are grouped into species based on the number of hidden neurons, which is indicative of the complexity of the search space. This speciation principle is indivisibly coupled with a novel genotype representation which is characterised by zero genome redundancy, high resilience to bloat, explicit marking of recurrent connections, as well as an efficient and reproducible stack-based evaluation procedure for networks with arbitrary topology. Furthermore, the proposed speciation principle is employed in several techniques designed to promote and preserve diversity within species and in the ecosystem as a whole. The competitive performance of the proposed framework, named Cortex, is demonstrated through experiments. A highly customisable software platform which implements the concepts proposed in this study is also introduced in the hope that it will serve as a useful and reliable tool for experimentation in the field of neuroevolution.

We train an evaluation function for a multi-player Chess variant board game called Duchess, which is played between two teams of two or three players, and is similar to Chess but with extra pieces, larger board size and significantly greater branching factor. Leaf positions in the alpha-beta search tree are evaluated with a linear combination of features, whose values are trained by self-play using the TreeStrap algorithm. We find superior performance can be achieved with an incremental approach, where the material values are learned first, followed by the attacking and defending values, and finally the piece-square values. To speed up the search, we evaluate board positions in a cumulative manner - identifying only those features that have changed, compared to the position at the previous move, and adjusting the evaluation accordingly.

Abstract Algorithms for evolving agents that learn during their lifetime have typically been evaluated on only a handful of environments. Designing such environments is labour intensive, potentially biased, and provides only a small sample size that may prevent accurate general conclusions from being drawn. In this paper we introduce a method for automatically generating MDP environments which allows the difficulty to be scaled in several ways. We present a case study in which environments are generated that vary along three key dimensions of difficulty: the number of environment configurations, the number of available actions, and the length of each trial. The study reveals interesting differences between three neural network models -- Fixed-Weight, Plastic-Weight, and Modulated Plasticity -- that would not have been obvious without sweeping across these different dimensions. Our paper thus introduces a new way of conducting reinforcement learning science: instead of manually designing a few environments, researchers will be able to automatically generate a range of environments across key dimensions of variation. This will allow scientists to more rigorously assess the general learning capabilities of an algorithm, and may ultimately improve the rate at which we discover how to create AI with general purpose learning.

Evolutionary systems such as Learning Classifier Systems (LCS) are able to learn reliably in irregular domains, while Artificial Neural Networks (ANNs) are very successful on problems with an appropriate gradient. This study introduces a novel method for discovering coarse structure, using a technique related to LCS, in combination with gradient descent. The structure used is a deep feature network, with a number of properties of a higher level of abstraction than existing ANNs, for example the network is constructed based on co-occurrence relationships, and maintained as a dynamic population of features. The feature creation technique can be considered a coarse or rapid initialization technique, that constructs a network before subsequent fine-tuning using gradient descent. The process is comparable with, but distinct from, layer-wise pretraining methods that construct and initialize a deep network prior to fine-tuning. The approach we introduce is a general learning technique, with assumptions of the dimensionality of input, and the described method uses convolved features. Results of classification of MNIST images show an average error rate of 0.79% without pre-processing or pretraining, comparable to the benchmark result provided by Restricted Boltzmann Machines of 0.95%, and 0.79% using dropout, however based on a convolutional topology, and as such our system is less general than RBM techniques, but more general than existing convolutional systems because it does not require the same domain assumptions and pre-defined topology. Use of a randomly initialized network provides a much poorer result (1.25%) indicating the coarse learning process plays a significant role. Classification of NORB images is examined, with results comparable to SVM approaches. Development of higher level relationships between features using this approach offers a distinct method of learning using a deep network of features, that can be used in combination with existing techniques.

We describe a new programming language called HERCL, designed for evolutionary computation with the specific aim of allowing new programs to be created by combining patches of code from different parts of other programs, at multiple scales. Large-scale patches are followed up by smaller-scale patches or mutations, recursively, to produce a global random search strategy known as hierarchical evolutionary re-combination. We demonstrate the proposed system on the task of learning to encode with the Caesar or Vigenere Cipher, and show how the evolution of one task may fruitfully be cross-pollinated with evolved solutions from other related tasks.

In traditional computer role-playing games (CRPGs), vendors are merchants who trade with players. Until now, these games have used vendors with static pricing or extremely simple pricing models, and this has hindered player immersion. In an effort to solve this problem, we present Crafty, a tool allowing developers to easily implement dynamic pricing mechanics for their games' vendors. Crafty studies player demand and chooses prices so as to maximise the profit. It thus allows CRPGs to be populated by vendors with intelligence, personality and intent, while requiring minimal effort by developers. Play-testing results show that players perceive such vendors more as fellow characters than as vending machines, improving the experience of the role-play.

Internal symmetry networks are a recently developed class of cellular neural network inspired by the phenomenon of internal symmetry in quantum physics. Their hidden unit activations are acted on non-trivially by the dihedral group of symmetries of the square. Here, we extend Internal symmetry networks to include recurrent connections, and train them by backpropagation to perform two simple image processing tasks.

In this paper we introduce a new algorithm for updating the parameters of a heuristic evaluation function, by updating the heuristic towards the values computed by an alpha-beta search. Our algorithm differs from previous approaches to learning from search, such as Samuels checkers player and the TD-Leaf algorithm, in two key ways. First, we update all nodes in the search tree, rather than a single node. Second, we use the outcome of a deep search, instead of the outcome of a subsequent search, as the training signal for the evaluation function. We implemented our algorithm in a chess program Meep, using a linear heuristic function. After initialising its weight vector to small random values, Meep was able to learn high quality weights from self-play alone. When tested online against human opponents, Meep played at a master level, the best performance of any chess program with a heuristic learned entirely from self-play.

We demonstrate applicability of a general class of multivariate probability density functions of the form e-P(x), where P(x) is an elliptic polynomial, to decentralised data fusion tasks. In particular, we derive an extension to the covariance Intersect algorithm for this class of distributions and demonstrate the necessary operations - diffusion, multiplication and linear transformation - for Bayesian operations. A simulated target tracking application demonstrates the use of these operations in a decentralised scenario, employing range-only sensing to show their generality beyond Gaussian representations.

This paper compares the efficacy of different crossover operators for Grammatical Evolution across a typical numeric regression problem and a typical data classification problem. Grammatical evolution is an extension of genetic programming, in that it is an algorithm for evolving complete programs in an arbitrary language. Each of the two main crossover operators struggles (for different reasons) to achieve 100% correct solutions. A mechanism is proposed, allowing the evolutionary algorithm to self-select the type of crossover utilised and this is shown to improve the rate of generating 100% successful solutions.

Grammatical evolution is an extension of genetic programming, in that it is an algorithm for evolving complete programs in an arbitrary language. By utilising a Backus Naur form grammar the advantages of typing are achieved as well as a separation of genotype and phenotype. This paper introduces a meta-grammar into grammatical evolution allowing the grammar to dynamically define functions, self-adaptively at the individual level without the need for special purpose operators or constraints. The user need not determine the architecture of the dynamically defined functions. As the search proceeds through genotype/phenotype space the number and use of the functions can vary. The ability of the grammar to dynamically define such functions allows regularities in the problem space to be exploited even where such regularities were not apparent when the problem was set up.

A new reinforcement learned neural network, that follows the ideas of the minibrain network but includes exploration and learns through both positive and negative feedback, is proposed. The proposed ReL network is evaluated against the minibrain network in the n × n grid world domain and the taxi domain and is shown to perform significantly better than the minibrain network.

Grammatical evolution is an algorithm for evolving complete programs in an arbitrary language. By utilising a Backus Naur Form grammar the advantages of typing are achieved. A separation of genotype and phenotype allows the implementation of operators that manipulate (for instance by crossover and mutation) the genotype (in grammatical evolution - a sequence of bits) irrespective of the genotype to phenotype mapping (in grammatical evolution $an arbitrary grammar). This paper introduces a new type of crossover operator for grammatical evolution. The crossover operator uses information automatically extracted from the grammar to minimise any destructive impact from the crossover. The information, which is extracted at the same time as the genome is initially decoded, allows the swapping between entities of complete expansions of non-terminals in the grammar without disrupting useful blocks of code on either side of the two point crossover. In the domains tested, results confirm that the crossover is (i) more productive than hill-climbing; (ii) enables populations to continue to evolve over considerable numbers of generations without intron bloat; and (iii) allows populations (in the domains tested) to reach higher fitness levels, quicker.

In this paper, we present a real-time algorithm that plans mostly optimal trajectories for multiple mobile robots in a dynamic environment. This approach combines the use of a Delaunay triangulation to discretise the environment, a novel efficient use of the A* search method, and a novel cubic spline representation for a robot trajectory that meets the kinematic and dynamic constraints of the robot. We show that for complex environments the shortest-distance path is not always the shortest-time path due to these constraints. The algorithm has been implemented on real robots, and we present experimental results in cluttered environments.

We present a new paradigm extending the iterated prisoner's dilemma to multiple players. Our model is unique in granting players information about past interactions between all pairs of players - allowing for much more sophisticated social behaviour. We provide an overview of preliminary results and discuss the implications in terms of the evolutionary dynamics of strategies.

As a test bed for studying evolutionary and other machine learning techniques, we have developed a simulated hockey game called Shock in which players attempt to shoot a puck into their enemy's goal during a fixed time period. Multiple players may participate-one can be controlled by a human user, while the others are guided by artificial controllers. In previous work, we introduced the Shock environment and presented players that received global input (as if from an overhead camera) and were trained on a restricted task, using an evolutionary hill climbing algorithm, with a staged learning approach (A. Blair and E. Sklar, 1998). Here, we expand upon this work by developing players which instead receive input from local, Braitenberg-style sensors (V. Braitenberg, 1984). These players are able to learn the task with fewer restrictions, using a simpler fitness measure based purely on whether or not a goal was scored. Moreover, they evolve to develop robust strategies for moving around the rink and scoring goals.

Following Tesauro's work on TD-Gammon, we used a 4,000 parameter feedforward neural network to develop a competitive backgammon evaluation function. Play proceeds by a roll of the dice, application of the network to all legal moves, and selection of the position with the highest evaluation. However, no backpropagation, reinforcement or temporal difference learning methods were employed. Instead we apply simple hillclimbing in a relative fitness environment. We start with an initial champion of all zero weights and proceed simply by playing the current champion network against a slightly mutated challenger and changing weights if the challenger wins. Surprisingly, this worked rather well. We investigate how the peculiar dynamics of this domain enabled a previously discarded weak method to succeed, by preventing suboptimal equilibria in a “meta-game” of self-learning.

Pollack (1991) demonstrated that second-order recurrent neural networks can act as dynamical recognizers for formal languages when trained on positive and negative examples, and observed both phase transitions in learning and interacted function system-like fractal state sets. Follow on work focused mainly on the extraction and minimization of a finite state automaton (FSA) from the trained network. However, such networks are capable of inducing languages that are not regular and therefore not equivalent to any FSA. Indeed, it may be simpler for a small network to fit its training data by inducing such a nonregular language. But when is the network's language not regular? In this article, using a low-dimensional network capable of learning all the Tomita data sets, we present an empirical method for testing whether the language induced by the network is regular. We also provide a detailed "-machine analysis of trained networks for both regular and nonregular languages.