Abstract The primary contribution of this work is the identification of the equivalence between imprecise probabilities and quantum probabilities, resulting in the development of the concept of probability waves. Probability waves serve as a straightforward mathematical tool for generalizing the most prevalent description of imprecise knowledge, which entails an interval with lower and upper bounds denoted by $$\underline{p}(y)$$ p ̲ ( y ) and $$\overline{p}(y)$$ p ¯ ( y ) . Probability waves establish a connection between imprecise probabilities and contemporary theories of quantum cognition, which elucidate how humans make decisions for uncertain events. The Bayesian interpretation of probability waves facilitates the identification of instances where knowledge of distribution parameters is insufficient to make any decision.

A vast majority of the current research in the field of machine learning is done using algorithms with strong arguments pointing to their biological implausibility such as backpropagation, deviating the field's focus from understanding its original organic inspiration to a compulsive search for optimal performance. Yet there have been a few proposed models that respect most of the biological constraints present in the human brain and are valid candidates for mimicking some of its properties and mechanisms. In this letter, we focus on guiding the learning of a biologically plausible generative model called the Helmholtz machine in complex search spaces using a heuristic based on the human image perception mechanism. We hypothesize that this model's learning algorithm is not fit for deep networks due to its Hebbian-like local update rule, rendering it incapable of taking full advantage of the compositional properties that multilayer networks provide. We propose to overcome this problem by providing the network's hidden layers with visual queues at different resolutions using multilevel data representation. The results on several image data sets showed that the model was able to not only obtain better overall quality but also a wider diversity in the generated images, corroborating our intuition that using our proposed heuristic allows the model to take more advantage of the network's depth growth. More important, they show the unexplored possibilities underlying brain-inspired models and techniques.

In computer vision research, convolutional neural networks (CNNs) have demonstrated remarkable capabilities at extracting patterns from raw pixel data, achieving state-of-the-art recognition accuracy. However, they significantly differ from human visual perception, prioritizing pixel-level correlations and statistical patterns, often overlooking object semantics. To explore this difference, we propose an approach that isolates core visual features crucial for human perception and object recognition: color, texture, and shape. In experiments on three benchmarks-Fruits 360, CIFAR-10, and Fashion MNIST-each visual feature is individually input into a neural network. Results reveal data set-dependent variations in classification accuracy, highlighting that deep learning models tend to learn pixel-level correlations instead of fundamental visual features. To validate this observation, we used various combinations of concatenated visual features as input for a neural network on the CIFAR-10 data set. CNNs excel at learning statistical patterns in images, achieving exceptional performance when training and test data share similar distributions. To substantiate this point, we trained a CNN on CIFAR-10 data set and evaluated its performance on the "dog" class from CIFAR-10 and on an equivalent number of examples from the Stanford Dogs data set. The CNN poor performance on Stanford Dogs images underlines the disparity between deep learning and human visual perception, highlighting the need for models that learn object semantics. Specialized benchmark data sets with controlled variations hold promise for aligning learned representations with human cognition in computer vision research.

Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns [34], this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory-decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning.

One of the most well established brain principles, Hebbian learning, has led to the theoretical concept of neural assemblies. Based on it, many interesting brain theories have spawned. Palm's work implements this concept through multiple binary Willshaw associative memories, in a model that not only has a wide cognitive explanatory power but also makes neuroscientific predictions. Yet, Willshaw's associative memory can only achieve top capacity when the stored vectors are extremely sparse (number of active bits can grow logarithmically with the vector's length). This strict requirement makes it difficult to apply any model that uses this associative memory, like Palm's, to real data. Hence the fact that most works apply the memory to optimal randomly generated codes that do not represent any information. This issue creates the need for encoders that can take real data, and produce sparse representations - a problem which is also raised following Barlow's efficient coding principle. In this work, we propose a biologically-constrained network that encodes images into codes that are suitable for Willshaw's associative memory. The network is organized into groups of neurons that specialize on local receptive fields, and learn through a competitive scheme. After conducting auto- and hetero-association experiments on two visual data sets, we can conclude that our network not only beats sparse coding baselines, but also that it comes close to the performance achieved using optimal random codes.

Although deep learning has solved difficult problems in visual pattern recognition, it is mostly successful in tasks where there are lots of labeled training data available. Furthermore, the global back-propagation based training rule and the amount of employed layers represents a departure from biological inspiration. The brain is able to perform most of these tasks in a very general way from limited to no labeled data. For these reasons it is still a key research question to look into computational principles in the brain that can help guide models to unsupervisedly learn good representations which can then be used to perform tasks like classification. To that end, we start by recalling four key brain-inspired principles that relate to simple vision: modeling ”whats” and ”wheres” separately; including a time component; context dependency; and layer-wise learning. Then, we take these principles and use them to convey an a priori structure to our model that makes the learning problem easier. With that, our model is able to generate such high quality representations for the MNIST data set. We compare the obtained results with similar recent works and verify extremely competitive results.

Convolutional neural networks (CNNs) evolved from Fukushima's neocognitron model, which is based on the ideas of Hubel and Wiesel about the early stages of the visual cortex. Unlike other branches of neocognitron-based models, the typical CNN is based on end-to-end supervised learning by backpropagation and removes the focus from built-in invariance mechanisms, using pooling not as a way to tolerate small shifts but as a regularization tool that decreases model complexity. These properties of end-to-end supervision and flexibility of structure allow the typical CNN to become highly tuned to the training data, leading to extremely high accuracies on typical visual pattern recognition data sets. However, in this work, we hypothesize that there is a flip side to this capability, a hidden overfitting. More concretely, a supervised, backpropagation based CNN will outperform a neocognitron/map transformation cascade (MTC) when trained and tested inside the same data set. Yet if we take both models trained and test them on the same task but on another data set (without retraining), the overfitting appears. Other neocognitron descendants like the What-Where model go in a different direction. In these models, learning remains unsupervised, but more structure is added to capture invariance to typical changes. Knowing that, we further hypothesize that if we repeat the same experiments with this model, the lack of supervision may make it worse than the typical CNN inside the same data set, but the added structure will make it generalize even better to another one. To put our hypothesis to the test, we choose the simple task of handwritten digit classification and take two well-known data sets of it: MNIST and ETL-1. To try to make the two data sets as similar as possible, we experiment with several types of preprocessing. However, regardless of the type in question, the results align exactly with expectation.

Willshaw networks are single-layered neural networks that store associations between binary vectors. Using only binary weights, these networks can be implemented efficiently to store large numbers of patterns and allow for fault-tolerant recovery of those patterns from noisy cues. However, this is only the case when the involved codes are sparse and randomly generated. In this letter, we use a recently proposed approach that maps visual patterns into informative binary features. By doing so, we manage to transform MNIST handwritten digits into well-distributed codes that we then store in a Willshaw network in autoassociation. We perform experiments with both noisy and noiseless cues and verify a tenuous impact on the recovered pattern's relevant information. More specifically, we were able to perform retrieval after filling the memory to several factors of its number of units while preserving the information of the class to which the pattern belongs.

We investigate exact indexing for high dimensional l p norms based on the 1-Lipschitz property and projection operators. The orthogonal projection that satisfies the 1-Lipschitz property for the l p norm is described. The adaptive projection defined by the first principal component is introduced.

It is believed that energy efficiency is an important constraint in brain evolution. As synaptic transmission dominates energy consumption, energy can be saved by ensuring that only a few synapses are active. It is therefore likely that the formation of sparse codes and sparse connectivity are fundamental objectives of synaptic plasticity. In this work we study how sparse connectivity can result from a synaptic learning rule of excitatory synapses. Information is maximised when potentiation and depression are balanced according to the mean presynaptic activity level and the resulting fraction of zero-weight synapses is around 50%. However, an imbalance towards depression increases the fraction of zero-weight synapses without significantly affecting performance. We show that imbalanced plasticity corresponds to imposing a regularising constraint on the L1-norm of the synaptic weight vector, a procedure that is well-known to induce sparseness. Imbalanced plasticity is biophysically plausible and leads to more efficient synaptic configurations than a previously suggested approach that prunes synapses after learning. Our framework gives a novel interpretation to the high fraction of silent synapses found in brain regions like the cerebellum.

The subsymbolic representation of the world often corresponds to a pattern that mirrors the world as described by the biological sense organs. Sparse binary vectors can describe subsymbolic representations, which can be efficiently stored in associative memories. According to the production system theory, a geometrically based problem-solving model can be defined as a production system operating on subsymbols. Our goal is to form a sequence of associations, which lead to a desired state represented by subsymbols, from an initial state represented by subsymbols. A simple and universal heuristic function can be defined, which takes into account the relationship between the vector and the corresponding similarity of the represented object or state in the real world. The manipulation of the subsymbols is described by a simple proto logic, which verifies if a subset of subsymbols is present in a set of subsymbols.

Abstract: We present a new architecture of a diagnostic system composed of an associative memory with feedback connections. This new architecture requires limited computer resources, it is fast, and it can run on small computers. The diagnostic process is described by a deduction system that performs an abductive inference. The abductive inference itself is explained by the verbal category theory. We model both processes by an associative memory that performs the inference with the aid of the feedback connections. The represented knowledge is arranged in groups that define taxonomy. An embedded diagnostic system for the determination of a disorder with applications in modern industrial machines is presented.

Abstract: Expert systems can be used to determine some objects or consequences from uncertain knowledge by hierarchical categorization. Categorical representation is psychologically motivated and also offers an explanation of how to deal with uncertain knowledge based on counting during approximate reasoning. It is an alternative to other well‐known uncertainty calculi. A knowledge base which is used during approximate reasoning is represented by a taxonomical arrangement of verbal categories. Priming eases the formation of the final hypothesis, as more exact possible hypotheses are formed. The approximate reasoning is demonstrated on an expert system ‘Jurassic’ from the field of paleontology for the determination of a dinosaur species. It helps the paleontologist to determine creatures from uncertain knowledge. The system is composed of 423 rules arranged in a directed acyclic graph with a depth of 5. This knowledge is represented by a taxonomical arrangement of verbal categories represented by associative memories.

We introduce a biologically and psychologically plausible neuronal model which could explain how pictorial reasoning is carried out by the human brain. This biologically inspired model throws some light on how some problem-solving abilities might actually be performed by the human brain using neural cell assemblies. It also highlights the benefits of distributed representation. These benefits include the ability to learn from experience, heuristics resulting from picture representation and the ability to deal with noisy information. The description of the reasoning process by formation of associations gives flexibility and power in operation. The proper way to use it, the proper data representations and the proper network architecture are presented in this paper.