• Neuromimetic metaplasticity enables catastrophic forgetting-free adaptive learning. • The metaplasticity model replicates the features of biological working memory. • A capacity-performance tradeoff is achieved without pre- or post-processing. • Frequency-dependent consolidation selectively filters erroneous memories. Conventional intelligent systems based on deep neural network (DNN) models encounter challenges in achieving human-like continual learning due to catastrophic forgetting. Here, we propose a metaplasticity model inspired by human working memory, enabling DNNs to perform catastrophic forgetting-resistant continual learning without any pre- or post-processing. A key aspect of our approach involves implementing distinct types of synapses from stable to flexible, and randomly intermixing them to train synaptic connections with different degrees of flexibility. This strategy allowed the network to successfully learn a continuous stream of information, even under unexpected changes in input length. The model achieved a balanced tradeoff between memory capacity and performance without requiring additional training or structural modifications, dynamically allocating memory resources to retain both old and new information. Furthermore, the model demonstrated robustness against data poisoning attacks by selectively filtering out erroneous memories, leveraging the Hebb repetition effect to reinforce the retention of significant data.

The brain prepares for learning even before interacting with the environment, by refining and optimizing its structures through spontaneous neural activity that resembles random noise. However, the mechanism of such a process has yet to be understood, and it is unclear whether this process can benefit the algorithm of machine learning. Here, we study this issue using a neural network with a feedback alignment algorithm, demonstrating that pretraining neural networks with random noise increases the learning efficiency as well as generalization abilities without weight transport. First, we found that random noise training modifies forward weights to match backward synaptic feedback, which is necessary for teaching errors by feedback alignment. As a result, a network with pre-aligned weights learns notably faster and reaches higher accuracy than a network without random noise training, even comparable to the backpropagation algorithm. We also found that the effective dimensionality of weights decreases in a network pretrained with random noise. This pre-regularization allows the network to learn simple solutions of a low rank, reducing the generalization error during subsequent training. This also enables the network to robustly generalize a novel, out-of-distribution dataset. Lastly, we confirmed that random noise pretraining reduces the amount of meta-loss, enhancing the network ability to adapt to various tasks. Overall, our results suggest that random noise training with feedback alignment offers a straightforward yet effective method of pretraining that facilitates quick and reliable learning without weight transport.

Long-range horizontal connections (LRCs) are conspicuous anatomical structures in the primary visual cortex (V1) of mammals, yet their detailed functions in relation to visual processing are not fully understood. Here, we show that LRCs are key components to organize a "small-world network" optimized for each size of the visual cortex, enabling the cost-efficient integration of visual information. Using computational simulations of a biologically inspired model neural network, we found that sparse LRCs added to networks, combined with dense local connections, compose a small-world network and significantly enhance image classification performance. We confirmed that the performance of the network appeared to be strongly correlated with the small-world coefficient of the model network under various conditions. Our theoretical model demonstrates that the amount of LRCs to build a small-world network depends on each size of cortex and that LRCs are beneficial only when the size of the network exceeds a certain threshold. Our model simulation of various sizes of cortices validates this prediction and provides an explanation of the species-specific existence of LRCs in animal data. Our results provide insight into a biological strategy of the brain to balance functional performance and resource cost.

Neurons in the primary visual cortex (V1) are often classified as simple or complex cells, but it is debated whether they are discrete hierarchical classes of neurons or if they represent a continuum of variation within a single class of cells. Herein, we show that simple and complex cells may arise commonly from the feedforward projections from the retina. From analysis of the cortical receptive fields in cats, we show evidence that simple and complex cells originate from the periodic variation of ON-OFF segregation in the feedforward projection of retinal mosaics, by which they organize into periodic clusters in V1. From data in cats, we observed that clusters of simple and complex receptive fields correlate topographically with orientation maps, which supports our model prediction. Our results suggest that simple and complex cells are not two distinct neural populations but arise from common retinal afferents, simultaneous with orientation tuning.

The brain successfully performs visual object recognition with a limited number of hierarchical networks that are much shallower than artificial deep neural networks (DNNs) that perform similar tasks. Here, we show that long-range horizontal connections (LRCs), often observed in the visual cortex of mammalian species, enable such a cost-efficient visual object recognition in shallow neural networks. Using simulations of a model hierarchical network with convergent feedforward connections and LRCs, we found that the addition of LRCs to the shallow feedforward network significantly enhances the performance of networks for image classification, to a degree that is comparable to much deeper networks. We found that a combination of sparse LRCs and dense local connections dramatically increases performance per wiring cost. From network pruning with gradient-based optimization, we also confirmed that LRCs could emerge spontaneously by minimizing the total connection length while maintaining performance. Ablation of emerged LRCs led to a significant reduction of classification performance, which implies these LRCs are crucial for performing image classification. Taken together, our findings suggest a brain-inspired strategy for constructing a cost-efficient network architecture to implement parsimonious object recognition under physical constraints such as shallow hierarchical depth.

The serial-position effect in working memory is considered important for studying how a sequence of sensory information can be retained and manipulated simultaneously in neural memory circuits. Here, via a precise analysis of the primacy and recency effects in human psychophysical experiments, we propose that stable and flexible codings take distinct roles of retaining and updating information in working memory, and that their combination induces serial-position effects spontaneously. We found that stable encoding retains memory to induce the primacy effect, while flexible encoding used for learning new inputs induces the recency effect. A model simulation based on human data, confirmed that a neural network with both flexible and stable synapses could reproduce the major characteristics of serial-position effects. Our new prediction, that the control of resource allocation by flexible-stable coding balance can modulate memory performance in sequence-specific manner, was supported by pre-cued memory performance data in humans.

Synchronized gamma frequency oscillations in neural networks are thought to be important to sensory information processing, and their effects have been intensively studied. Here we describe a mechanism by which the nervous system can readily control gamma oscillation effects, depending selectively on visual stimuli. Using a model neural network simulation, we found that sensory response in the primary visual cortex is significantly modulated by the resonance between "spontaneous" and "stimulus-driven" oscillations. This gamma resonance can be precisely controlled by the synaptic plasticity of thalamocortical connections, and cortical response is regulated differentially according to the resonance condition. The mechanism produces a selective synchronization between the afferent and downstream neural population. Our simulation results explain experimental observations such as stimulus-dependent synchronization between the thalamus and the cortex at different oscillation frequencies. The model generally shows how sensory information can be selectively routed depending on its frequency components.

Synchronized oscillation is very commonly observed in many neuronal systems and might play an important role in the response properties of the system. We have studied how the spontaneous oscillatory activity affects the responsiveness of a neuronal network, using a neural network model of the visual cortex built from Hodgkin-Huxley type excitatory (E-) and inhibitory (I-) neurons. When the isotropic local E-I and I-E synaptic connections were sufficiently strong, the network commonly generated gamma frequency oscillatory firing patterns in response to random feed-forward (FF) input spikes. This spontaneous oscillatory network activity injects a periodic local current that could amplify a weak synaptic input and enhance the network's responsiveness. When E-E connections were added, we found that the strength of oscillation can be modulated by varying the FF input strength without any changes in single neuron properties or interneuron connectivity. The response modulation is proportional to the oscillation strength, which leads to self-regulation such that the cortical network selectively amplifies various FF inputs according to its strength, without requiring any adaptation mechanism. We show that this selective cortical amplification is controlled by E-E cell interactions. We also found that this response amplification is spatially localized, which suggests that the responsiveness modulation may also be spatially selective. This suggests a generalized mechanism by which neural oscillatory activity can enhance the selectivity of a neural network to FF inputs.