Research has shown that neurons within the brain are selective to certain stimuli. For example, the fusiform face area (FFA) region is known by neuroscientists to selectively activate when people see faces over non-face objects. While the exact mechanisms by which the primary visual system directs information to the correct higher levels of the brain are currently unknown, there are high-level neural mechanisms of perception that we can incorporate in a novel computational model - ones that utilizes lateral and top down feedback in the form of hierarchical competition. We demonstrate that these neural mechanisms provide the foundation of a novel classification framework that rivals traditional supervised learning in computer vision. Additionally, we show that the innate priors built into our architecture support out of distribution generalization on the application of face detection.

Deep neural networks (DNNs) are easily fooled by adversarial perturbations that are impercep-tible to humans. Adversarial training, a process where adversarial examples are added to the training set, is the current state-of-the-art defense against adversarial attacks, but it lowers the model's accuracy on clean inputs, is computationally expensive, and offers less robustness to natural noise. In contrast, energy-based models (EBMs), which were designed for efficient im-plementation in neuromorphic hardware and physical systems, incorporate feedback connections from each layer to the previous layer, yielding a recurrent, deep-attractor architecture which we hypothesize should make them naturally robust. Our work is the first to explore the robustness of EBMs to both natural cor-ruptions and adversarial attacks, which we do using the CIFAR-IO and CIFAR-IOO datasets. We demonstrate that EBMs are more robust than transformers and display comparable robustness to adversarially-trained DNNs on gradient-based (white-box) attacks, query-based (black-box) attacks, and natural perturbations without sacrificing clean accuracy, and without the need for adversarial training or additional training techniques.

Although Convolutional Neural Networks (CNNs) achieve high accuracy on image recognition tasks, they lack robustness against realistic corruptions and fail catastrophically when deliberately attacked. Previous CNNs with representations similar to primary visual cortex (V1) were more robust to adversarial attacks on images than current adversarial defense techniques, but they required training on large-scale neural recordings or handcrafting neuroscientific models. Motivated by evidence that neural activity in V1 is sparse, we develop a class of hybrid CNNs, called LCANets, which feature a frontend that performs sparse coding via local lateral competition. We demonstrate that LCANets achieve competitive clean accuracy to standard CNNs on action and image recognition tasks and significantly greater accuracy under various image corruptions. We also perform the first adversarial attacks with full knowledge of a sparse coding CNN layer by attacking LCANets with white-box and black-box attacks, and we show that, contrary to previous hypotheses, sparse coding layers are not very robust to white-box attacks. Finally, we propose a way to use sparse coding layers as a plug-and-play robust frontend by showing that they significantly increase the robustness of adversarially-trained CNNs over corruptions and attacks.

Previous work has shown that commonly-used metrics for comparing representations between neural networks overestimate similarity due to correlations between data points. We show that intra-example feature correlations also causes significant overestimation of network similarity and propose an image inversion technique to analyze only the features used by a network. With this technique, we find that similarity across architectures is significantly lower than commonly understood, but we surprisingly find that similarity between models with different architectures increases as the adversarial robustness of the models increase. Our findings indicate that robust networks tend toward a universal set of representations, regardless of architecture, and that the robust training criterion is a strong prior constraint on the functions that can be learned by diverse modern architectures. We also find that the representations learned by a robust network of any architecture have an asymmetric overlap with non-robust networks of many architectures, indicating that the representations used by robust neural networks are highly entangled with the representations used by non-robust networks.

Exploration of novel computational platforms is critical for the advancement of artificial intelligence as we approach the physical limitations of traditional hardware. Biologically accurate, energy efficient neuromorphic systems are particularly promising for enabling future breakthroughs because of their ability to process information in parallel and to scale using extremely low power. Sparse coding is a signal processing technique which has been known to model the information encoding in the primary visual cortex. When sparse solutions are solved using local neuron competition along with the unsupervised dictionary learning that mimics cortical development, we can build an end to end, hardware to software, brain inspired solution to a machine learning problem. In this article, we perform a detailed comparison of sparse coding solutions generated classically by orthogonal matching pursuit (OMP) implemented on a conventional digital processor with spike-based solutions obtained using the Intel Loihi neuromorphic processor. A novel “post-hoc” normalization technique to shorten simulation time for Loihi is presented along with analysis of optimal parameter selection, reconstruction errors, and unsupervised dictionary learning for Loihi approaches and their classical counterparts. Preliminary results show that both the Loihi full simulation approach and the post-hoc normalization approach are well suited to neuromorphic processors and operate in a size, weight and power regime that is not accessible by classical approaches. Ultimately, the use of this normalization technique allows for faster and, often, better solutions than demonstrated previously.

Efforts to improve the learning abilities of neural networks have focused mostly on the role of optimization methods rather than on weight initializations. Recent findings, however, suggest that neural networks rely on lucky random initial weights of subnetworks called “lottery tickets” that converge quickly to a solution [1]. To investigate how weight initializations affect performance, we examine small convolutional networks that are trained to predict$n$steps of the two-dimensional cellular automaton Conway's Game of Life, the update rules of which can be implemented efficiently in a small CNN. We find that networks of this architecture trained on this task rarely converge. Rather, networks require substantially more parameters to consistently converge. Furthermore, we find that the initialization parameters that gradient descent converges to a solution are sensitive to small perturbations, such as a single sign change. Finally, we observe a critical value$d_{0}$such that training minimal networks with examples in which cells are alive with probability$d_{0}$dramatically increases the chance of convergence to a solution. Our results are consistent with the lottery ticket hypothesis [1].

Adversarial examples for neural network image classifiers are known to be transferable: examples optimized to be misclassified by a source classifier are often misclassified as well by classifiers with different architectures. However, targeted adversarial examples—optimized to be classified as a chosen target class—tend to be less transferable between architectures. While prior research on constructing transferable targeted attacks has focused on improving the optimization procedure, in this work we examine the role of the source classifier. Here, we show that training the source classifier to be "slightly robust"—that is, robust to small-magnitude adversarial examples—substantially improves the transferability of class-targeted and representation-targeted adversarial attacks, even between architectures as different as convolutional neural networks and transformers. The results we present provide insight into the nature of adversarial examples as well as the mechanisms underlying so-called "robust" classifiers.

While deep learning continues to permeate through all fields of signal processing and machine learning, a critical exploit in these frameworks exists and remains unsolved. These exploits, or adversarial examples, are a type of signal attack that can change the output class of a classifier by perturbing the stimulus signal by an imperceptible amount. The attack takes advantage of statistical irregularities within the training data, where the added perturbations can move the image across deep learning decision boundaries. What is even more alarming is the transferability of these attacks to different deep learning models and architectures. This means a successful attack on one model has adversarial effects on other, unrelated models. In a general sense, adversarial attack through perturbations is not a machine learning vulnerability. Human and biological vision can also be fooled by various methods, i.e. mixing high and low frequency images together, by altering semantically related signals, or by sufficiently distorting the input signal. However, the amount and magnitude of such a distortion required to alter biological perception is at a much larger scale. In this work, we explored this gap through the lens of biology and neuroscience in order to understand the robustness exhibited in human perception. Our experiments show that by leveraging sparsity and modeling the biological mechanisms at a cellular level, we are able to mitigate the effect of adversarial alterations to the signal that have no perceptible meaning. Furthermore, we present and illustrate the effects of top-down functional processes that contribute to the inherent immunity in human perception in the context of exploiting these properties to make a more robust machine vision system.

Deep feed-forward convolutional neural networks (CNNs) have become ubiquitous in virtually all machine learning and computer vision challenges; however, advancements in CNNs have arguably reached an engineering saturation point where incremental novelty results in minor performance gains. Although there is evidence that object classification has reached human levels on narrowly defined tasks, for general applications, the biological visual system is far superior to that of any computer. Research reveals there are numerous missing components in feed-forward deep neural networks that are critical in mammalian vision. The brain does not work solely in a feed-forward fashion, but rather all of the neurons are in competition with each other; neurons are integrating information in a bottom up and top down fashion and incorporating expectation and feedback in the modeling process. Furthermore, our visual cortex is working in tandem with our parietal lobe, integrating sensory information from various modalities.

BACKGROUND: Histopathology images of tumor biopsies present unique challenges for applying machine learning to the diagnosis and treatment of cancer. The pathology slides are high resolution, often exceeding 1GB, have non-uniform dimensions, and often contain multiple tissue slices of varying sizes surrounded by large empty regions. The locations of abnormal or cancerous cells, which may constitute a small portion of any given tissue sample, are not annotated. Cancer image datasets are also extremely imbalanced, with most slides being associated with relatively common cancers. Since deep representations trained on natural photographs are unlikely to be optimal for classifying pathology slide images, which have different spectral ranges and spatial structure, we here describe an approach for learning features and inferring representations of cancer pathology slides based on sparse coding. RESULTS: We show that conventional transfer learning using a state-of-the-art deep learning architecture pre-trained on ImageNet (RESNET) and fine tuned for a binary tumor/no-tumor classification task achieved between 85% and 86% accuracy. However, when all layers up to the last convolutional layer in RESNET are replaced with a single feature map inferred via a sparse coding using a dictionary optimized for sparse reconstruction of unlabeled pathology slides, classification performance improves to over 93%, corresponding to a 54% error reduction. CONCLUSIONS: We conclude that a feature dictionary optimized for biomedical imagery may in general support better classification performance than does conventional transfer learning using a dictionary pre-trained on natural images.

Hierarchical networks are known to achieve high classification accuracy on difficult machine-learning tasks. For many applications, a clear explanation of why the data was classified a certain way is just as important as the classification itself. However, the complexity of hierarchical networks makes them ill-suited for existing explanation methods. We propose a new method, contribution propagation, that gives per-instance explanations of a trained network's classifications. We give theoretical foundations for the proposed method, and evaluate its correctness empirically. Finally, we use the resulting explanations to reveal unexpected behavior of networks that achieve high accuracy on visual object-recognition tasks using well-known data sets.

Wireless networks of biologically inspired distributed sensors (BIDS) are hypothesized to enable improved overall detection accuracy using ultra-low power and low bandwidth spike-based communication between nodes. Unlike traditional sensor networks, in which nodes communicate via digital protocols that require precise decoding of binary signal packets, BIDS nodes communicate by broadcasting generic radio frequency pulses, or spikes. Individual BIDS nodes are modeled after leaky integrate-and-fire (LIF) neurons, in which both filtered sensory signals and inputs from other BIDS nodes are accumulated as capacitive charge that decays with a characteristic time constant. A BIDS node itself broadcasts a spike whenever its internal state exceeds a threshold value. Here we present detailed simulations of a BIDS network designed to detect a moving target-modeled as a pure acoustic tone with a translating origin-against a background of 1/f noise. In the absence of a target, the average internal state is well below threshold and noise-induced spikes recruit little additional activity. In contrast, the presence of a target pushes the average internal state closer to threshold, such that each spike is now able to recruit additional spikes, leading to a chain reaction. Our results show that while individual BIDS nodes may be noisy and unreliable, a network of BIDS nodes is capable of highly reliable detection even when the signal-to-noise ratio (SNR) on individual nodes is low. We demonstrate that collective computation between nodes supports improved detection accuracy in a manner that is extremely robust to the damage or loss of individual nodes.

We investigate the role of learned shape-prototypes in an influential family of hierarchical neural-network models of vision. Central to these networks' design is a dictionary of learned shapes, which are meant to respond to discriminative visual patterns in the input. While higher-level features based on such learned prototypes have been cited as key for viewpoint-invariant object-recognition in these models [1], [2], we show that high performance on invariant object-recognition tasks can be obtained by using a simple set of unlearned, “shape-free” features. This behavior is robust to the size of the network. These results call into question the roles of learning and shape-specificity in the success of such models on difficult vision tasks, and suggest that randomly constructed prototypes may provide a useful “universal” dictionary.

In the primate's primary visual cortex (V1), cells are classified in terms of two categories: simple cells and complex cells, given their response properties. While simple cells respond strongly to gating and bar stimuli at a certain phase and location, responses of complex cells are insensitive to small translation of stimulus within the receptive field [1]. Inspired by the response properties of simple and complex cells in the primary visual cortex, we propose a computational network to learn the receptive fields of these cells, and address the development of translation invariance from a temporal sequence of natural images. A generative model with sparseness constraints is devised to minimize the energy of prediction errors. Each simple cell is modulated by a higher layer of complex cells in a multiplicative fashion, where a slowness property and a trace-like rule are enforced on complex cells, as the result of a temporal coherence soft constraint. Furthermore, non-negativity constraints of the latent cell variables and weight matrices are imposed to fit the known neurophysiology. We present an online gradient descent algorithm to train our model from natural image sequences, in which a pre-training strategy is used to initialize the weights. The developed connection weights show that complex cell outputs are directly proportional to quadratic forms of simple cell responses. Each receptive field of simple cells develop a Gabor-like orientation filter, and each complex cell pools similar simple cell receptive fields - in retinotopic and feature space - producing the locally-invariant representation.

Conventional sparse coding learns optimal dictionaries of feature bases to approximate input signals; however, it is not favorable to classify the inputs. Recent research has focused on building discriminative sparse coding models to facilitate the classification tasks. In this paper, we develop a new discriminative sparse coding model via bidirectional flows. Sensory inputs (from bottom-up) and discriminative signals (supervised from top-down) are propagated through a hierarchical network to form sparse representations at each level. The ℓ0-constrained sparse coding model allows highly efficient online learning and does not require iterative steps to reach a fixed point of the sparse representation. The introduction of discriminative top-down information flows helps to group reconstructive features belonging to the same class and thus to benefit the classification tasks. Experiments are conducted on multiple data sets including natural images, hand-written digits and 3-D objects with favorable results. Compared with unsupervised sparse coding via only bottom-up directions, the two-way discriminative approach improves the recognition performance significantly.

Can lateral connectivity in the primary visual cortex account for the time dependence and intrinsic task difficulty of human contour detection? To answer this question, we created a synthetic image set that prevents sole reliance on either low-level visual features or high-level context for the detection of target objects. Rendered images consist of smoothly varying, globally aligned contour fragments (amoebas) distributed among groups of randomly rotated fragments (clutter). The time course and accuracy of amoeba detection by humans was measured using a two-alternative forced choice protocol with self-reported confidence and variable image presentation time (20-200 ms), followed by an image mask optimized so as to interrupt visual processing. Measured psychometric functions were well fit by sigmoidal functions with exponential time constants of 30-91 ms, depending on amoeba complexity. Key aspects of the psychophysical experiments were accounted for by a computational network model, in which simulated responses across retinotopic arrays of orientation-selective elements were modulated by cortical association fields, represented as multiplicative kernels computed from the differences in pairwise edge statistics between target and distractor images. Comparing the experimental and the computational results suggests that each iteration of the lateral interactions takes at least [Formula: see text] ms of cortical processing time. Our results provide evidence that cortical association fields between orientation selective elements in early visual areas can account for important temporal and task-dependent aspects of the psychometric curves characterizing human contour perception, with the remaining discrepancies postulated to arise from the influence of higher cortical areas.

Cortical neurons selective for numerosity may underlie an innate number sense in both animals and humans. We hypothesize that the number- selective responses of cortical neurons may in part be extracted from coherent, object-specific oscillations . Here, indirect evidence for this hypothesis is obtained by analyzing the numerosity information encoded by coherent oscillations in artificially generated spikes trains. Several experiments report that gamma-band oscillations evoked by the same object remain coherent, whereas oscillations evoked by separate objects are uncorrelated. Because the oscillations arising from separate objects would add in random phase to the total power summed across all stimulated neurons, we postulated that the total gamma activity, normalized by the number of spikes, should fall roughly as the square root of the number of objects in the scene, thereby implicitly encoding numerosity. To test the hypothesis, we examined the normalized gamma activity in multiunit spike trains, 50 to 1000 msec in duration, produced by a model feedback circuit previously shown to generate realistic coherent oscillations. In response to images containing different numbers of objects, regardless of their shape, size, or shading, the normalized gamma activity followed a square-root-of-n rule as long as the separation between objects was sufficiently large and their relative size and contrast differences were not too great. Arrays of winner-take-all numerosity detectors, each responding to normalized gamma activity within a particular band, exhibited tuning curves consistent with behavioral data. We conclude that coherent oscillations in principle could contribute to the number-selective responses of cortical neurons, although many critical issues await experimental resolution.

A population coded algorithm, built on established models of motion processing in the primate visual system, computes the time-to-collision of a mobile robot to real-world environmental objects from video imagery. A set of four transformations starts with motion energy, a spatiotemporal frequency based computation of motion features. The following processing stages extract image velocity features similar to, but distinct from, optic flow; "translation" features, which account for velocity errors including those resulting from the aperture problem; and finally, estimate the time-to-collision. Biologically motivated population coding distinguishes this approach from previous methods based on optic flow. A comparison of the population coded approach with the popular optic flow algorithm of Lucas and Kanade against three types of approaching objects shows that the proposed method produces more robust time-to-collision information from a real world input stimulus in the presence of the aperture problem and other noise sources. The improved performance comes with increased computational cost, which would ideally be mitigated by special purpose hardware architectures.

Synchronous firing limits the amount of information that can be extracted by averaging the firing rates of similarly tuned neurons. Here, we show that the loss of such rate-coded information due to synchronous oscillations between retinal ganglion cells can be overcome by exploiting the information encoded by the correlations themselves. Two very different models, one based on axon-mediated inhibitory feedback and the other on oscillatory common input, were used to generate artificial spike trains whose synchronous oscillations were similar to those measured experimentally. Pooled spike trains were summed into a threshold detector whose output was classified using Bayesian discrimination. For a threshold detector with short summation times, realistic oscillatory input yielded superior discrimination of stimulus intensity compared to rate-matched Poisson controls. Even for summation times too long to resolve synchronous inputs, gamma band oscillations still contributed to improved discrimination by reducing the total spike count variability, or Fano factor. In separate experiments in which neurons were synchronized in a stimulus-dependent manner without attendant oscillations, the Fano factor increased markedly with stimulus intensity, implying that stimulus-dependent oscillations can offset the increased variability due to synchrony alone.

High-frequency oscillatory potentials (HFOPs) in the vertebrate retina are stimulus specific. The phases of HFOPs recorded at any given retinal location drift randomly over time, but regions activated by the same stimulus tend to remain phase locked with approximately zero lag, whereas regions activated by spatially separate stimuli are typically uncorrelated. Based on retinal anatomy, we previously postulated that HFOPs are mediated by feedback from a class of axon-bearing amacrine cells that receive excitation from neighboring ganglion cells-via gap junctions-and make inhibitory synapses back onto the surrounding ganglion cells. Using a computer model, we show here that such circuitry can account for the stimulus specificity of HFOPs in response to both high- and low-contrast features. Phase locking between pairs of model ganglion cells did not depend critically on their separation distance, but on whether the applied stimulus created a continuous path between them. The degree of phase locking between spatially separate stimuli was reduced by lateral inhibition, which created a buffer zone around strongly activated regions. Stimulating the inhibited region between spatially separate stimuli increased their degree of phase locking proportionately. Our results suggest several experimental strategies for testing the hypothesis that stimulus-specific HFOPs arise from axon-mediated feedback in the inner retina.

Moving stimuli elicit oscillatory responses from retinal ganglion cells at frequencies between 60-100 Hz. We used a computer model of the inner retina to investigate whether the additional firing synchrony resulting from stimulus-evoked high frequency oscillations could contribute to the detection of moving bars. The responses of the model ganglion cells were similar to those of cat alpha cells. Event trains from the model ganglion cells simulated by moving bars were summed into a threshold detector with short integration window (2-4 msec) whose output was classified by an ideal observer. To isolate the contribution from firing correlations, the model ganglion cells were replaced by independent Poisson generators with matched time-dependent event rates. Compared to this control, firing correlations between the model ganglion cells allowed for improved detection of moving stimuli.