Recurrent neural networks (RNNs) are powerful models used widely in both machine learning and neuroscience to learn tasks with temporal dependencies and to model neural dynamics. However, despite significant advancements in the theory of RNNs, there is still limited understanding of their learning process and the impact of the temporal structure of data. Here, we bridge this gap by analyzing the learning dynamics of linear RNNs (LRNNs) analytically, enabled by a novel framework that accounts for task dynamics. Our mathematical result reveals four key properties of LRNNs: (1) Learning of data singular values is ordered by both scale and temporal precedence, such that singular values that are larger and occur later are learned faster. (2) Task dynamics impact solution stability and extrapolation ability. (3) The loss function contains an effective regularization term that incentivizes small weights and mediates a tradeoff between recurrent and feedforward computation. (4) Recurrence encourages feature learning, as shown through a novel derivation of the neural tangent kernel for finite-width LRNNs. As a final proof-of-concept, we apply our theoretical framework to explain the behavior of LRNNs performing sensory integration tasks. Our work provides a first analytical treatment of the relationship between the temporal dependencies in tasks and learning dynamics in LRNNs, building a foundation for understanding how complex dynamic behavior emerges in cognitive models.

Does the process of training a neural network to solve a task tend to use all of the available weights even when the task could be solved with fewer weights? To address this question we study the effects of pruning fully connected, convolutional and residual models while varying their widths. We find that the proportion of weights that can be pruned without degrading performance is largely invariant to model size. Increasing the width of a model has little effect on the density of the pruned model relative to the increase in absolute size of the pruned network. In particular, we find substantial prunability across a large range of model sizes, where our biggest model is 50 times as wide as our smallest model. We explore three hypotheses that could explain these findings. Source code: [29].

In-context learning (ICL) is one of the most powerful and most unexpected capabilities to emerge in recent transformer-based large language models (LLMs). Yet the mechanisms that underlie it are poorly understood. In this paper, we demonstrate that comparable ICL capabilities can be acquired by an alternative sequence prediction learning method using clone-structured causal graphs (CSCGs). Moreover, a key property of CSCGs is that, unlike transformer-based LLMs, they are {\em interpretable}, which considerably simplifies the task of explaining how ICL works. Specifically, we show that it uses a combination of (a) learning template (schema) circuits for pattern completion, (b) retrieving relevant templates in a context-sensitive manner, and (c) rebinding of novel tokens to appropriate slots in the templates. We go on to marshall evidence for the hypothesis that similar mechanisms underlie ICL in LLMs. For example, we find that, with CSCGs as with LLMs, different capabilities emerge at different levels of overparameterization, suggesting that overparameterization helps in learning more complex template (schema) circuits. By showing how ICL can be achieved with small models and datasets, we open up a path to novel architectures, and take a vital step towards a more general understanding of the mechanics behind this important capability.

We characterise the problem of abstraction in the context of deep reinforcement learning. Various well established approaches to analogical reasoning and associative memory might be brought to bear on this issue, but they present difficulties because of the need for end-to-end differentiability. We review developments in AI and machine learning that could facilitate their adoption.

Human perception and experience of time are strongly influenced by ongoing stimulation, memory of past experiences, and required task context. When paying attention to time, time experience seems to expand; when distracted, it seems to contract. When considering time based on memory, the experience may be different than what is in the moment, exemplified by sayings like "time flies when you're having fun." Experience of time also depends on the content of perceptual experience-rapidly changing or complex perceptual scenes seem longer in duration than less dynamic ones. The complexity of interactions among attention, memory, and perceptual stimulation is a likely reason that an overarching theory of time perception has been difficult to achieve. Here, we introduce a model of perceptual processing and episodic memory that makes use of hierarchical predictive coding, short-term plasticity, spatiotemporal attention, and episodic memory formation and recall, and apply this model to the problem of human time perception. In an experiment with approximately 13,000 human participants, we investigated the effects of memory, cognitive load, and stimulus content on duration reports of dynamic natural scenes up to about 1 minute long. Using our model to generate duration estimates, we compared human and model performance. Model-based estimates replicated key qualitative biases, including differences by cognitive load (attention), scene type (stimulation), and whether the judgment was made based on current or remembered experience (memory). Our work provides a comprehensive model of human time perception and a foundation for exploring the computational basis of episodic memory within a hierarchical predictive coding framework.

We present a slot-wise, object-based transition model that decomposes a scene into objects, aligns them (with respect to a slot-wise object memory) to maintain a consistent order across time, and predicts how those objects evolve over successive frames. The model is trained end-to-end without supervision using transition losses at the level of the object-structured representation rather than pixels. Thanks to the introduction of our novel alignment module, the model deals properly with two issues that are not handled satisfactorily by other transition models, namely object persistence and object identity. We show that the combination of an object-level loss and correct object alignment over time enables the model to outperform a state-of-the-art baseline, and allows it to deal well with object occlusion and re-appearance in partially observable environments.

Robust perception relies on both bottom-up and top-down signals. Bottom-up signals consist of what’s directly observed through sensation. Top-down signals consist of beliefs and expectations based on past experience and the current reportable short-term memory, such as how the phrase ‘peanut butter and ...’ will be completed. The optimal combination of bottom-up and top-down information remains an open question, but the manner of combination must be dynamic and both context and task dependent. To effectively utilize the wealth of potential top-down information available, and to prevent the cacophony of intermixed signals in a bidirectional architecture, mechanisms are needed to restrict information flow. We explore deep recurrent neural net architectures in which bottom-up and top-down signals are dynamically combined using attention. Modularity of the architecture further restricts the sharing and communication of information. Together, attention and modularity direct information flow, which leads to reliable performance improvements in perceptual and language tasks, and in particular improves robustness to distractions and noisy data. We demonstrate on a variety of benchmarks in language modeling, sequential image classification, video prediction and reinforcement learning that the \emph{bidirectional} information flow can improve results over strong baselines.

With a view to bridging the gap between deep learning and symbolic AI, we present a novel end-to-end neural network architecture that learns to form propositional representations with an explicitly relational structure from raw pixel data. In order to evaluate and analyse the architecture, we introduce a family of simple visual relational reasoning tasks of varying complexity. We show that the proposed architecture, when pre-trained on a curriculum of such tasks, learns to generate reusable representations that better facilitate subsequent learning on previously unseen tasks when compared to a number of baseline architectures. The workings of a successfully trained model are visualised to shed some light on how the architecture functions.

We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is agnostic to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries and can adapt in continuously changing environments. In our policy consolidation model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent’s policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.

One of the main concerns of deep reinforcement learning (DRL) is the data inefficiency problem, which stems both from an inability to fully utilize data acquired and from naive exploration strategies. In order to alleviate these problems, we propose a DRL algorithm that aims to improve data efficiency via both the utilization of unrewarded experiences and the exploration strategy by combining ideas from unsupervised auxiliary tasks, intrinsic motivation, and hierarchical reinforcement learning (HRL). Our method is based on a simple HRL architecture with a metacontroller and a subcontroller. The subcontroller is intrinsically motivated by the metacontroller to learn to control aspects of the environment, with the intention of giving the agent: 1) a neural representation that is generically useful for tasks that involve manipulation of the environment and 2) the ability to explore the environment in a temporally extended manner through the control of the metacontroller. In this way, we reinterpret the notion of pixel- and feature-control auxiliary tasks as reusable skills that can be learned via an intrinsic reward. We evaluate our method on a number of Atari 2600 games. We found that it outperforms the baseline in several environments and significantly improves performance in one of the hardest games-Montezuma's revenge-for which the ability to utilize sparse data is key. We found that the inclusion of intrinsic reward is crucial for the improvement in the performance and that most of the benefit seems to be derived from the representations learned during training.

Deep neural networks excel at function approximation, yet they are typically trained from scratch for each new function. On the other hand, Bayesian methods, such as Gaussian Processes (GPs), exploit prior knowledge to quickly infer the shape of a new function at test time. Yet, GPs are computationally expensive, and it can be hard to design appropriate priors. In this paper we propose a family of neural models, Conditional Neural Processes (CNPs), that combine the benefits of both. CNPs are inspired by the flexibility of stochastic processes such as GPs, but are structured as neural networks and trained via gradient descent. CNPs make accurate predictions after observing only a handful of training data points, yet scale to complex functions and large datasets. We demonstrate the performance and versatility of the approach on a range of canonical machine learning tasks, including regression, classification and image completion.

Unlike humans, who are capable of continual learning over their lifetimes, artificial neural networks have long been known to suffer from a phenomenon known as catastrophic forgetting, whereby new learning can lead to abrupt erasure of previously acquired knowledge. Whereas in a neural network the parameters are typically modelled as scalar values, an individual synapse in the brain comprises a complex network of interacting biochemical components that evolve at different timescales. In this paper, we show that by equipping tabular and deep reinforcement learning agents with a synaptic model that incorporates this biological complexity (Benna & Fusi, 2016), catastrophic forgetting can be mitigated at multiple timescales. In particular, we find that as well as enabling continual learning across sequential training of two simple tasks, it can also be used to overcome within-task forgetting by reducing the need for an experience replay database.

We present a novel neural episodic memory architecture that utilizes reservoir computing to extract and recall information gleaned over time from a multilayer perceptron that receives sensory input. Reservoir computing models project input data into a high-dimensional dynamical space and also serve as a fading memory that holds on to past inputs thereby enabling the direct association of the current input with the past. The architecture presented utilizes these capabilities via an abstract feedback mechanism and in doing so creates attractor-like states within the reservoir that are associated with each discrete memory and associates these states and therefore memories over time into episodes. In addition, the feedback mechanism provides stabilization to an otherwise chaotic complex dynamical system.

A significant obstacle in using phenomenological models of spiking neurons for large-scale simulations is the approximation of the optimal parameters for a type of neuron, given the available experimental data. Here we show a method for optimizing the parameters of such models, based on a combination of different frequency-current and voltage-current relations of a neuron as well as known physiological properties. We also present a python toolbox which uses NeMo spiking neural network simulator and provides a fast GPU-based implementation of our method. As a benchmark, our toolbox was used to fit Izhikevich equations to neurological data obtained from a cat's thalamic relay cell. Our resulting model was able to predict the firing patterns of known membrane potential traces of this neuron, although they were not explicitly defined during training. A further comparison between this neuron model and a previous approach, when both models are used in the simulation of a generic thalamic nucleus, revealed that the distribution of neuronal avalanches is significantly different and conforms better to power law-like distributions, thus increasing the likelihood of a critical regime and the biological plausibility of the simulation.

Neuronal avalanches are a local cortical phenomenon characterised by bursts of activity bracketed by periods of quiescence. It has been shown both in vivo and in vitro that these avalanches exhibit features of systems within a critical state. Locally critical system's avalanches conform to power law-like distributions. Globally these systems consist of modules exhibiting long-range temporal correlations identifiable via Detrended Fluctuation Analysis (DFA). Using an eight module oscillatory spiking neural network we analyse the correlation between these local and global criticality markers. Our findings demonstrate that locally critical modules promote long-range temporal correlations. Furthermore, when local modules are no longer critical we find that modules become uncorrelated or noisy. This suggests a strong link between local and global critical behaviour.

Low-frequency oscillations have been the target of extensive research both in cortical structures and in the basal ganglia, due to numerous reports of associations with brain disorders and the normal functioning of the brain. Whereas a number of computational models of the basal ganglia investigate these phenomena, these models tend to focus on intrinsic oscillatory mechanisms, neglecting evidence that points to the cortex as the origin of this oscillatory behaviour. In this work we constructed a neural model of the basal ganglia circuitry and used it to investigate the relationship between frequency of oscillatory cortical input, dopamine and the effectiveness of the basal ganglia as an action selection device. Our simulations show the impact of the phase offset between different cortical inputs. This was found to be highly dependent on the frequency band and to have a strong influence on basal ganglia effectiveness. In addition, the level of dopamine in the system was found to modulate this effect, also depending on the input's frequency band.

Neuronal avalanches are a cortical phenomenon characterised by bursts of activity bracketed by periods of quiescence. It has been shown both in vivo and in vitro that the size and length of avalanche events conform to power law-like distributions, suggesting the system is within or near a critical state. This work investigates the interplay of network connectivity, synaptic plasticity, and criticality. Using two different network construction algorithms, we demonstrate that Spike Timing Dependent Plasticity (STDP) robustly drives the network towards a critical state. Our findings show that, while the initial distribution of synaptic weights plays a significant role in attaining criticality, the network's topology at the local level has little or no impact.

It has been proposed that oscillating groups of neurons firing synchronously provide a mechanism that underlies many cognitive functions. In previous work, it has been demonstrated that, in a network of excitatory and inhibitory neurons, a synchronous response gradually emerges due to spike timing dependant plasticity (STDP) acting upon an external spatio-temporal stimulus that is repeatedly applied. This paper builds on these findings by addressing two questions relating to STDP and network dynamics in the context of pyramidal inter-neuronal gamma (PING) network architectures. Firstly, how does the choice of neuron model affect the learning of oscillation through STDP? Our experiments suggest that the earlier results hinge on the selection of a simple, biologically less realistic neuron model. Secondly, how do neural oscillators that have learned to oscillate only in response to a particular stimulus behave when connected to other such neural oscillators? We investigate this question in the context of a network of PING oscillators, in order to understand the relationship between coupling strength and the onset of synchrony, emulating the results of a classic experiment by Kuramoto.

It has been proposed that groups of neurons firing synchronously provide a mechanism that underlies many cognitive functions such as attention, associative learning, and working memory, as well as opening up communication channels between neuron groups. A mathematical abstraction that is gaining increasing acceptance for modeling neural information processing is the Kuramoto oscillator model, which can be used as an elementary unit to represent populations of oscillatory neurons. Whilst the Kuramoto model is widely used to capture fundamental properties of the collective dynamics of interacting communities of oscillatory neurons, the question arises as to how well it performs this role. This paper aims to address that question experimentally by using neural models to replicate the most fundamental of Kuramoto's findings, in which he showed that for any number of oscillators there is a critical coupling value Kcbelow which the oscillators are fully unsynchronized and another critical coupling value KL≥ Kcabove wich all oscillators become fully sunchronized. In this study, we replace Kuramoto oscillators with oscillating polulations both of quadratic integrate-and-fire neurons and of Hodgkin-Huxley neurons to establish whether Kuramoto's findings still hold in a more biologically realistic setup. The individual oscillators use a pyramidal inter-neuronal gamma architecture designed using a novel evolutionary technique.

A novel clustering algorithm is presented for analyzing the temporal dynamics of synchronization in networks of coupled oscillators and applied to a model of resting-state brain activity. Connectivity in the model is based on a human-brain structural connectivity matrix derived from diffusion tensor imaging tractography. We find a strong correspondence between areas of high synchronization and highly connected “hub” nodes, anatomical regions forming the structural core of the network linking all areas of the brain. Such models have the potential to increase our understanding of the constraints placed on brain function by underlying anatomical structure.

In this paper, we present a spiking neural network architecture that autonomously learns to control a 4 degree-of-freedom robotic arm after an initial period of motor babbling. Its aim is to provide the joint commands that will move the end-effector in a desired spatial direction, given the joint configuration of the arm. The spiking neurons have been simulated according to Izhikevich's model, which exhibits biologically realistic behaviour and yet is computationally efficient. The architecture is a feed-forward network where the input layers encode the intended movement direction of the end-effector in spatial coordinates, as well as the information that is given by proprioception about the current joint angles of the arm. The motor commands are determined by decoding the firing patterns in the output layers. Both excitatory and inhibitory synapses connect the input and output layers, and their initial weights are set to random values. The network learns to map input stimuli to motor commands during a phase of repetitive action-perception cycles, in which Spike Timing-Dependent Plasticity (STDP) strengthens synapses between neurons that are correlated and weakens synapses between uncorrelated ones. The trained spiking neural network has been successfully tested on a kinematic model of the arm of an iCub humanoid robot.

Spiking neural network simulators provide environments in which to implement and experiment with models of biological brain structures. Simulating large-scale models is computationally expensive, however, due to the number and interconnectedness of neurons in the brain. Furthermore, where such simulations are used in an embodied setting, the simulation must be real-time in order to be useful. In this paper we present a platform (nemo) for such simulations which achieves high performance on parallel commodity hardware in the form of graphics processing units (GPUs). This work makes use of the Izhikevich neuron model which provides a range of realistic spiking dynamics while being computationally efficient. Learning is facilitated through spike-timing dependent synaptic plasticity. Our GPU kernel can deliver up to 550 million spikes per second using a single device. This corresponds to a real-time simulation of around 55 000 neurons under biologically plausible conditions with 1000 synapses per neuron and a mean firing rate of 10 Hz.

Simulating spiking neural networks is of great interest to scientists wanting to model the functioning of the brain. However, large-scale models are expensive to simulate due to the number and interconnectedness of neurons in the brain. Furthermore, where such simulations are used in an embodied setting, the simulation must be real-time in order to be useful. In this paper we present NeMo, a platform for such simulations which achieves high performance through the use of highly parallel commodity hardware in the form of graphics processing units (GPUs). NeMo makes use of the Izhikevich neuron model which provides a range of realistic spiking dynamics while being computationally efficient. Our GPU kernel can deliver up to 400 million spikes per second. This corresponds to a real-time simulation of around 40 000 neurons under biologically plausible conditions with 1000 synapses per neuron and a mean firing rate of 10 Hz.

This paper introduces an edge-based object recognition method that is robust with respect to clutter, occlusion and object deformations. The method combines the use of local features and their spatial relationships to identify the point correspondences between the object-of-interest and the input scene. Local features encode information from their neighbourhood, and this renders them insensitive to noise at a distance. However, they have moderate discriminating power, and the proposed method exploits their spatial structure to compensate for this. Our flexible localisation technique, which is based on point distribution models, makes the method also applicable to deformable objects. The point matching task is formulated as an optimisation problem that is solved using the Viterbi algorithm. The method has been validated on challenging real scenes.

Packing two-dimensional shapes on a surface such that no shapes overlap and the uncovered surface area is minimized is an important problem that arises in a variety of industrial applications. This paper introduces an intelligent system which tackles the most difficult instance of this problem, where two-dimensional irregular shapes have to be packed on a regularly or irregularly shaped surface. The proposed system utilizes techniques not previously applied to packing, drawn from computer vision and artificial intelligence, and achieves high-quality solutions with short computational times. In addition, the system deals with complex shapes and constraints that occur in industrial applications, such as defective regions and irregularly shaped sheets. We evaluate the effectiveness and efficiency of the proposed method using 14 established benchmark problems that are available from the EURO Special Interest Group on Cutting and Packing.

Previous work on event calculus planning has viewed it as an abductive task, whose implementation through abductive logic programming reduces to partial order planning. By contrast, this paper presents a formal framework for tackling event calculus planning through satisfiability, in the style of Kautz and Selman. A provably correct conjunctive normal form encoding for event calculus planning problems is supplied, rendering them soluble by an off-the-shelf SAT solver.

A narrative is a course of real events about which we might have incomplete information. Formalisms for reasoning about action may be broadly divided into those which are narrative-based, such as the event calculus of Kowalski and Sergot, and those which reason on the level of hypothetical sequences of actions, in particular the situation calculus. This paper bridges the gap between these types of formalism by supplying a technique for linking incomplete narrative descriptions to situation calculus domain formulae written in the usual style using a Result function. Particular attention is given to actions with duration and overlapping actions. By illuminating the relationship between these two different styles of representation, the paper moves us one step closer to a full understanding of the space of all possible formalisms for reasoning about action.