In natural cognition, confidence is used to evaluate the quality of decisions and adapt one's behavior to the task at hand.For now, artificial agents lack this kind of metacognitive ability and interact with their environment in a purely reactive way.Inspired by recent findings about the cognitive modeling of confidence, we propose a novel architecture for sequential decision-making.It combines an evidence accumulation model with a metacognitive module that computes and exploits confidence to tune the decision process.The model has been assessed on a perceptual decision-making task, showing promises for more flexible artificial agents and a possible path towards artificial metacognition.

Complementary Learning Systems theory holds that intelligent agents need two learning systems. Semantic memory is encoded in the neocortex with dense, overlapping representations and acquires structured knowledge. Episodic memory is encoded in the hippocampus with sparse, pattern-separated representations and quickly learns the specifics of individual experiences. Recently, this duality between semantic and episodic memories has been challenged by predictive coding, a biologically plausible neural network model of the neocortex which was shown to have hippocampus-like abilities on auto-associative memory tasks. These results raise the question of the episodic capabilities of the neocortex and their relation to semantic memory. In this paper, we present such a predictive coding model of the neocortex and explore its episodic capabilities. We show that this kind of model can indeed recall the specifics of individual examples but only if it is trained on a small number of examples. The model is overfitted to these exemples and does not generalize well, suggesting that episodic memory can arise from semantic learning. Indeed, a model trained with many more examples loses its recall capabilities. This work suggests that individual examples can be encoded gradually in the neocortex using dense, overlapping representations but only in a limited number, motivating the need for sparse, pattern-separated representations as found in the hippocampus.

Neural Episodic Control is a powerful reinforcement learning framework that employs a differentiable dictionary to store non-parametric memories. It was inspired by episodic memory on the functional level, but lacks a direct theoretical connection to the associative memory models generally used to implement such a memory. We first show that the dictionary is an instance of the recently proposed Universal Hopfield Network framework. We then introduce a continuous approximation of the dictionary readout operation in order to derive two energy functions that are Lyapunov functions of the dynamics. Finally, we empirically show that the dictionary outperforms the Max separation function, which had previously been argued to be optimal, and that performance can further be improved by replacing the Euclidean distance kernel by a Manhattan distance kernel. These results are enabled by the generalization capabilities of the dictionary, so a novel criterion is introduced to disentangle memorization from generalization when evaluating associative memory models.

International audience

In this article, we propose a modular reinforcement learning (MRL) architecture that coordinates the competition and the cooperation between modules, and inspires, in a developmental approach, the generation of new modules in cases where new goals have been detected.We evaluate the effectiveness of our approach in a multiple-goal torus grid world.Results show that our approach has better performance than previous MRL methods in learning separate strategies for sub-goals, and reusing them for solving task-specific or unseen multi-goal problems, as well as maintaining the independence of the learning in each module.

International audience

Augmenting the representation of the current state of the external world with internal states corresponding to working and episodic memories has been proposed as a bioinspired solution to apply models of reinforcement learning to non-Markovian tasks. But, transposing these results to behavioral and experimental neuroscience, it is not completely clear how each of these memories can contribute to learning the augmented representations and when they must act in association for more complex tasks. Choosing an elementary implementation of these memories and experimental tasks of decision making in rodents, we explore these pivotal situations and make concrete the underlying mechanisms and criteria. We also specify cases where additional mechanisms must be envisaged.

This paper presents a new approach for learning transition function in state representation learning (SRL) for control. While state-of-the-art methods use different deterministic neural networks to learn forward and inverse state transition functions independently with auto-supervised learning, we introduce a bidirectional stochastic model to learn both transition functions. We aim at using the uncertainty of the model on its predictions as an intrinsic motivation for exploration to enhance the representation learning. More, using the same model to learn both transition functions allows sharing the parameters, which can reduce their number and should increase the embedding quality of the representation. We use a factored restricted Boltzmann machine (fRBM) based model, enhanced with dedicated structure for learning system dynamics and transitions with shared parameters. The presented work focuses on building the structure of the bidirectional transition model for unsupervised learning. Our fRBM structure is directly inspired from physics interactions between inputs and outputs in reinforcement learning framework. We compare different training algorithms for learning the model that must be able to predict observable random variables to be used in SRL framework. Our structure is not restricted to any type of observable, nevertheless in this paper we focus on learning dynamics from the OpenAI Gym environment Swinging Pendulum. We show that the proposed structure is able to learn bidirectional transition function and performs well in prediction task.

We discuss here the characteristics of a software environment appropriate for the development of a bio-inspired cognitive architecture, which can emulate the behavior of autonomous intelligent agents. First, it is reminded that, while the focus is often set on the more abstract aspects of cognitive abilities, studying the fundamental bases of intelligence that allow for autonomy is a prerequisite for well defined intelligent systems. Secondly, we highlight functional loops associating cerebral structures including the basal ganglia in the brain of most species along the evolution. They are dedicated to the organization of behavior under the constraint of reinforcement , corresponding in their simplest expression to the selection of action for survival. Lastly, concerning the simulation of such models, we describe a software environment to study such relations in a more controlled way than hardware implementations, by adapting a platform built on the top of a video game for the development of classical artificial intelligence models. We explain here how our neuronal model exhibits bodily and internal characteristics necessary for survival tasks and how these characteristics are plugged in the simulation platform. Some scenarios of survival are reported as an illustration of this environment.

The neurocomputational model described here proposes that two dimensions involved in computation of reward prediction errors i.e magnitude and time could be computed separately and later combined unlike traditional reinforcement learning models. The model is built on biological evidences and is able to reproduce various aspects of classical conditioning, namely, the progressive cancellation of the predicted reward, the predictive firing from conditioned stimuli, and delineation of early rewards by showing firing for sooner early rewards and not for early rewards that occur with a longer latency in accordance with biological data.

Artificial Neural Networks are often used as black boxes to implement behavioral functions, developed by trials and errors, fed with sensory inputs and controlled by some criteria of performance. This is the case for pavlovian conditioning where important sensory information is non ambiguous and where the error of prediction is to be minimized. These past years, taking into account critical conditioning behaviors entailed complexifying the neuronal functioning and learning rules. This resulted in networks still simple at the architectural level but with a dynamics difficult to master. Instead, we propose a new neuronal model using uniform and classical neuronal dynamics, with a more complex architecture based on recent findings in neuroscience. Results reported in this paper confirm the good behavior of the model and justify the complex architecture by the greater robustness and flexibility of the model.

Basal Ganglia, a group of sub-cortical neuronal nuclei in the brain, are commonly described as the neuronal substratum to Reinforcement Learning. Since the seminal work by Schultz [1], a huge amount of work has been done to deepen that analogy, from functional and anatomic points of view. Nevertheless, a noteworthy architectural hint has been hardly explored: the outstanding reduction of dimensionality from the input to the output of the basal ganglia. Bar-Gad et al. [2] have suggested that this transformation could correspond to a Principal Component Analysis but did not explore the full functional consequences of this hypothesis. In this paper, we propose to study this mechanism within a model more realistic from a computational neuroscience point of view. Particularly, we show its feasibility when the loop is closed, in the framework of Action Selection.

In this paper, dynamic neural fields (DNFs) are used to develop key features of a cortically-inspired computational module. Under the perspective of designing computational systems that can exhibit the flexibility and genericity of the cortical substrate, using neural field as the competition layer for self-organising modules has to be considered. However, despite the fact that they serve as a biologically-inspired model, applying DNFs to drive self-organisation is not straightforward. In order to address that issue, an original method for evaluating neural field equations is proposed, based on statistical measurements of the field behaviour in some scenarios. Limitations of classical neural field equations are then quantified, and an original field equation is proposed to overcome these difficulties. The performance of the proposed field model is discussed in comparison with some previously considered models, leading to the promotion of the proposed model as a suitable mean for processing competition in cortex-like computation for cognitive systems.

This paper presents three pruning algorithms based on optimal brain surgeon (OBS) and unit-optimal brain surgeon (unit-OBS). The first variant performs a backward selection by successively removing single weights from the input variables to the hidden units in a fully connected multilayer perceptron (MLP) for variable selection. The second one removes a subset of non-significant weights in one step. The last one combines the two properties presented above. Simulation results obtained on the Monk's problem illustrate the specificities of each method described in this paper according to the preserved variables and the preserved weights.

Hardware developments have led to the use of shared memory as an efficient parallel programming method. The main goals of the work reported here are to speed up executions and to decrease development time of parallel neural network implementations. To allow for such implementations, a library has been defined, as a bridge between neural networks and general purpose MIMD computer parallelisms.

Most models concerned with real-world applications can be improved in structuring data and incorporating knowledge about the domain. In our problem of radio electrical wave dying down prediction for mobile communication, a geographic database can be divided in contextual subsets, each representing an homogeneous domain where a predictive model performs better. More precisely, by clustering the input space, a predictive model (here a multilayer perceptron) can be trained on each subspace. Various unsupervised algorithms for clustering were evaluated (Kohonen's maps, Desieno's algorithm 1988, neural gas, growing neural gas, Buhmann's algorithm 1992) to obtain classes homogeneous enough to decrease the predictive error of the radio electrical wave prediction.

This paper presents a model of cooperation between representations of correlation in the associative cortex and goal oriented representations of action in the prefrontal cortex. The model is applied to robot navigation and gives a framework for addressing planning with numerical neural techniques.

A typical navigation task requires both declarative and non-declarative memory for respectively locus memorization and action selection. These dual memories are believed to be achieved in the central nervous systems by the hippocampus and the cortex. In the light of the functioning of each neuronal system, the paper proposes a model of interaction between these structures. The model is illustrated through a simple example of a navigation task.

Symbolic and connectionist tools are widely used in the artificial intelligence (AI) domain. Both approaches report weaknesses to address the full range of capabilities required by this domain, and are often combined to overcome some limitations, but no extensive efforts have been done to give a wider view of the problem. More precisely, many ad hoc hybrid models were built from symbolic and connectionist tools, but little was said about the choice, integration and combination of such tools. Such was the goal of the European ESPRIT project MIX. We report the main conclusions of this project, together with its results on a real-world application.

The neural net model TOM (temporal organization map) that we present in the paper is a new connectionist approach whose time representation is different from the one in classical temporal connectionist models. The architecture is neurobiologically inspired and is dedicated to sensory problems involving a temporal dimension. The basic idea of the TOM model is the propagation of an activity throughout the network whose elements are organized according to a map architecture. This propagation leads to a triggering of a sequence detection. We have applied this new kind of architecture to a spoken digit recognition problem. The results draw near to the results of the best hidden Markov model (HMM) techniques. The interest of such an architecture is its genericity and the possibility to merge several data flows in order to improve the classical performances of neural nets.

From classical connectionist and symbolic models, the new field of neurosymbolic integration has emerged, whose aim is to benefit from the advantages of both domains to model human perceptive and cognitive capabilities. To reach this goal, some strategies are envisaged, among which connectionist cognitive processing claims that these desired capabilities can emerge from pure neuronal structures and processes. This approach refers to the substratum of cognition, the human brain and gives rise to perceptually grounded models whose goal is to reach higher cognitive levels. Its principles are presented here and, as an illustration, an application to invariant pattern recognition is described. From basic connectionist models, a biologically inspired model of neuronal networks cooperation is implemented to allow for internal information translation. This mechanism leads to automatic pattern centring in a classical character recognition application with excellent performances.

In this paper, a semi-empirical model of field strength prediction combining theoretical results of propagation loss algorithms and artificial neural networks is considered. This approach expects to overcome some limitations inherent in existing semi-empirical models: linear behaviour of the statistical analysis used in the construction of the models, unfitness for learning new situations. The good results obtained in a dense urban area show that neural networks are a very efficient empirical method to compute new kinds of models which integrate theoretical and experimental data.

Presents the basic principles and some preliminary applications of a novel connectionist model for building up a neural network system. The model uses the cortical column instead of the classical neuron as the basic processing unit; it is closely related to the neurobiological modeling of the human cortex. The main features of this model which make it different from classical approaches concern the local connectivity and the integration of time by causality learning. The model provides a basic unit well -adapted to solve humanlike problems by integrating particular difficulties in its own structure (e.g. the coarticulation effect in speech recognition). The model has been applied to two difficult problems in the artificial intelligence field: speech recognition (more precisely, the acoustic-phonetic decoding of continuous speech) and biomedical X-ray image interpretation. The two systems that were designed for these applications demonstrate the ability of the cortical column to solve perceptive and cognitive tasks

The authors propose a novel approach to neuronlike models that consists in simulating cortical columns, i.e. associations of neurons having a specific, functional activity. This model is built according to a theory which is consistent with neurobiological data. The model is implemented through a network of columns that simulate the inherently parallel functioning of the nervous system. A description is given of this model, and the experimental conditions in which it is now being tested are described.>