The visual system of mammals is comprised of parallel, hierarchical specialized pathways. Different pathways are specialized in so far as they use representations that are more suitable for supporting specific downstream behaviours. In particular, the clearest example is the specialization of the ventral ("what") and dorsal ("where") pathways of the visual cortex. These two pathways support behaviours related to visual recognition and movement, respectively. To-date, deep neural networks have mostly been used as models of the ventral, recognition pathway. However, it is unknown whether both pathways can be modelled with a single deep ANN. Here, we ask whether a single model with a single loss function can capture the properties of both the ventral and the dorsal pathways. We explore this question using data from mice, who like other mammals, have specialized pathways that appear to support recognition and movement behaviours. We show that when we train a deep neural network architecture with two parallel pathways using a self-supervised predictive loss function, we can outperform other models in fitting mouse visual cortex. Moreover, we can model both the dorsal and ventral pathways. These results demonstrate that a self-supervised predictive learning approach applied to parallel pathway architectures can account for some of the functional specialization seen in mammalian visual systems.

Neurons in the dorsal visual pathway of the mammalian brain are selective for motion stimuli, with the complexity of stimulus representations increasing along the hierarchy. This progression is similar to that of the ventral visual pathway, which is well characterized by artificial neural networks (ANNs) optimized for object recognition. In contrast, there are no image-computable models of the dorsal stream with comparable explanatory power. We hypothesized that the properties of dorsal stream neurons could be explained by a simple learning objective: the need for an organism to orient itself during self-motion. To test this hypothesis, we trained a 3D ResNet to predict an agent's self-motion parameters from visual stimuli in a simulated environment. We found that the responses in this network accounted well for the selectivity of neurons in a large database of single-neuron recordings from the dorsal visual stream of non-human primates. In contrast, ANNs trained on an action recognition dataset through supervised or self-supervised learning could not explain responses in the dorsal stream, despite also being trained on naturalistic videos with moving objects. These results demonstrate that an ecologically relevant cost function can account for dorsal stream properties in the primate brain.

The sensory-motor neuron synapse of Aplysia is an excellent model system for investigating the biochemical changes underlying memory formation. In this system, training that is separated by rest periods (spaced training) leads to persistent changes in synaptic strength that depend on biochemical pathways that are different from those that occur when the training lacks rest periods (massed training). Recently, we have shown that in isolated sensory neurons, applications of serotonin, the neurotransmitter implicated in inducing these synaptic changes during memory formation, lead to desensitization of the PKC Apl II response, in a manner that depends on the method of application (spaced versus massed). Here, we develop a mathematical model of this response in order to gain insight into how neurons sense these different training protocols. The model was developed incrementally, and each component was experimentally validated, leading to two novel findings: First, the increased desensitization due to PKA-mediated heterologous desensitization is coupled to a faster recovery than the homologous desensitization that occurs in the absence of PKA activity. Second, the model suggests that increased spacing leads to greater desensitization due to the short half-life of a hypothetical protein, whose production prevents homologous desensitization. Thus, we predict that the effects of differential spacing are largely driven by the rates of production and degradation of proteins. This prediction suggests a powerful mechanism by which information about time is incorporated into neuronal processing.