Active inference is a formal approach to study cognition that views adaptive agents as engaged in a constant process of approximate Bayesian inference, via the minimisation of variational and expected free energies. In this way, agents are able to maximise the likelihood of preferred observations or states, given a generative model of the environment. While most implementations in the literature rely on expected free energy as an appropriate way to evaluate the desirability of policies, i.e., sequences of actions, there are different ways to consider the contribution of past motor experience to the agent’s future behaviour. In some approaches, agents are assumed to have knowledge of their own executed actions, and use it for planning future ones. In others, agents are unaware of their actions, and must infer their motor behaviour to plan for the future. This difference reflects a standard point of departure in two leading frameworks in motor control based on the presence, or not, of an efference copy signal conveying information about an agent’s own actions. In this work we compare the performances of action-aware and action-unaware agents in two navigations tasks, showing how action-unaware agents can achieve performances comparable to action-aware ones while at a severe disadvantage.

Emerging from the pairwise attention in conventional Transformers, there is a growing interest in sparse attention mechanisms that align more closely with localized, contextual learning in the biological brain. Existing studies such as the Coordination method employ iterative cross-attention mechanisms with a bottleneck to enable the sparse association of inputs. However, these methods are parameter inefficient and fail in more complex relational reasoning tasks. To this end, we propose Associative Transformer (AiT) to enhance the association among sparsely attended input tokens, improving parameter efficiency and performance in various vision tasks such as classification and relational reasoning. AiT leverages a learnable explicit memory comprising specialized priors that guide bottleneck attentions to facilitate the extraction of diverse localized tokens. Moreover, AiT employs an associative memory-based token reconstruction using a Hopfield energy function. The extensive empirical experiments demonstrate that AiT requires significantly fewer parameters and attention layers outperforming a broad range of sparse Transformer models. Additionally, AiT outperforms the SOTA sparse Transformer models including the Coordination method on the Sort-of-CLEVR dataset.

Conventional neural networks including Transformers encounter the catastrophic forgetting problem during sequential task learning, where learning new tasks interferes with previously learned knowledge. Existing memory replay and regularization methods cannot effectively eliminate interference among different tasks. Soft parameter sharing methods usually necessitate a large amount of additional parameters for learning each task, and task identity information is essential for leveraging task-specific parameters. To this end, we propose Remembering Transformer leveraging an adapter mixtures architecture enhanced by a generative routing mechanism for efficient task retention. In generative routing, input samples are allocated to the most relevant expert adapters based on a reconstruction loss. Moreover, unlike existing studies on soft parameter sharing that do not consider model capacity limitations, we investigate a challenging setting where the number of task-specific parameters is constrained. In particular, we devise an adapter fusion strategy to aggregate resembling experts based on similarity matching and knowledge distillation. Extensive empirical results measured by task accuracy, forgetting rate, and memory footprint, demonstrate that Remembering Transformer significantly enhances knowledge retention without task identity information. The proposed method surpasses various conventional methods with enhanced parameter efficiency in a broad range of incremental learning tasks.

Advancements in calcium indicators and optical techniques have made optical neural recording common in neuroscience. As data volumes grow, streamlining the analysis pipelines for image preprocessing, signal extraction, and subsequent neural activity analyses becomes essential. Challenges in analysis includes 1) ensuring data quality of original and processed data at each step, 2) selecting optimal algorithms and their parameters from numerous options, each with its own pros and cons, by implementing or installing them manually, 3) systematically recording each analysis step for reproducibility, and 4) adopting standard data formats for data sharing and meta-analyses. To address these challenges, we developed Optical Neuroimage Studio (OptiNiSt), a scalable, extendable, and reproducible framework for creating calcium data analysis pipelines. OptiNiSt includes the following features. 1) Researchers can easily create analysis pipelines by selecting multiple processing modules, tuning their parameters, and visualizing the results at each step through a graphic user interface in a web browser. 2) In addition to pre-installed tools, new analysis algorithms can be easily added. 3) Once a processing pipeline is designed, the entire workflow with its modules and parameters are stored in a YAML file, which makes the pipeline reproducible and deployable on high-performance computing clusters. 4) OptiNiSt can read image data in a variety of file formats and store the analysis results in NWB (Neurodata Without Borders), a standard data format for data sharing. We expect that this framework will be helpful in standardizing optical neural data analysis protocols.

The intersection between neuroscience and artificial intelligence (AI) research has created synergistic effects in both fields. While neuroscientific discoveries have inspired the development of AI architectures, new ideas and algorithms from AI research have produced new ways to study brain mechanisms. A well-known example is the case of reinforcement learning (RL), which has stimulated neuroscience research on how animals learn to adjust their behavior to maximize reward. In this review article, we cover recent collaborative work between the two fields in the context of meta-learning and its extension to social cognition and consciousness. Meta-learning refers to the ability to learn how to learn, such as learning to adjust hyperparameters of existing learning algorithms and how to use existing models and knowledge to efficiently solve new tasks. This meta-learning capability is important for making existing AI systems more adaptive and flexible to efficiently solve new tasks. Since this is one of the areas where there is a gap between human performance and current AI systems, successful collaboration should produce new ideas and progress. Starting from the role of RL algorithms in driving neuroscience, we discuss recent developments in deep RL applied to modeling prefrontal cortex functions. Even from a broader perspective, we discuss the similarities and differences between social cognition and meta-learning, and finally conclude with speculations on the potential links between intelligence as endowed by model-based RL and consciousness. For future work we highlight data efficiency, autonomy and intrinsic motivation as key research areas for advancing both fields.

This paper presents a new human activity recognition method that uses a camera mounted on a mobile robot. We assume that the robot's camera captures images of a person and recognizes his/her activities based on skeletal and visual features extracted from the images. A key issue encountered with this method for activity recognition is that it requires the robot to position itself so that it has an adequate field of view of the activities being conducted. For example, if the robot is directly behind a person while observing that person making tea, it will be difficult for the robot to distinguish that activity from other similar activities such as preparing a meal or washing dishes. Our method employs deep reinforcement learning to control the movements of the mobile robot that is observing the activities in order to maximize its recognition accuracy while minimizing its energy consumption related to its movement. We propose effective action- and state-space designs that can achieve early training convergence and highly accurate activity recognition by: (i) incorporating the confidence of the activity recognition output when evaluating the quality of the current state (position), (ii) incorporating the costs of subsequent actions when estimating values for those actions, and (iii) designing an effective action space that accelerates reinforcement learning by restricting the movement space of the robot to the circumference of a circle with a predefined radius centered on the person.

An important step in understanding the nature of the brain is to identify "cores" in the brain network, where brain areas strongly interact with each other. Cores can be considered as essential sub-networks for brain functions. In the last few decades, an information-theoretic approach to identifying cores has been developed. In this approach, interactions between parts are measured by an information loss function, which quantifies how much information would be lost if interactions between parts were removed. Then, a core called a "complex" is defined as a subsystem wherein the amount of information loss is locally maximal. Although identifying complexes can be a novel and useful approach, its application is practically impossible because computation time grows exponentially with system size. Here we propose a fast and exact algorithm for finding complexes, called Hierarchical Partitioning for Complex search (HPC). HPC hierarchically partitions systems to narrow down candidates for complexes. The computation time of HPC is polynomial, enabling us to find complexes in large systems (up to several hundred) in a practical amount of time. We prove that HPC is exact when an information loss function satisfies a mathematical property, monotonicity. We show that mutual information is one such information loss function. We also show that a broad class of submodular functions can be considered as such information loss functions, indicating the expandability of our framework to the class. We applied HPC to electrocorticogram recordings from a monkey and demonstrated that HPC revealed temporally stable and characteristic complexes.

We propose a curiosity reward based on information theory principles and consistent with the animal instinct to maintain certain critical parameters within a bounded range. Our experimental validation shows the added value of the additional homeostatic drive to enhance the overall information gain of a reinforcement learning agent interacting with a complex environment using continuous actions. Our method builds upon two ideas: i) To take advantage of a new Bellman-like equation of information gain and ii) to simplify the computation of the local rewards by avoiding the approximation of complex distributions over continuous states and actions.