Boundary Vector Cells (BVCs) are a class of neurons in the brains of vertebrates that encode environmental boundaries at specific distances and allocentric directions, playing a central role in forming place fields in the hippocampus. Most computational BVC models are restricted to two-dimensional (2D) environments, making them prone to spatial ambiguities in the presence of horizontal symmetries in the environment. To address this limitation, we incorporate vertical angular sensitivity into the BVC framework, thereby enabling robust boundary detection in three dimensions, and leading to significantly more accurate spatial localization in a biologically-inspired robot model.The proposed model processes LiDAR data to capture vertical contours, thereby disambiguating locations that would be indistinguishable under a purely 2D representation. Experimental results show that in environments with minimal vertical variation, the proposed 3D model matches the performance of a 2D baseline; yet, as 3D complexity increases, it yields substantially more distinct place fields and markedly reduces spatial aliasing. These findings show that adding a vertical dimension to BVC-based localization can significantly enhance navigation and mapping in real-world 3D spaces while retaining performance parity in simpler, near-planar scenarios.

Learning meaningful sentences is different from learning a random set of words. When humans understand the meaning, the learning occurs relatively quickly. What mechanisms enable this to happen? In this paper, we examine the learning of novel sequences in familiar situations. We embed the Small World of Words (SWOW-EN), a Word Association Norms (WAN) dataset, in a spiking neural network based on the Hierarchical Temporal Memory (HTM) model to simulate long-term memory. Results show that in the presence of SWOW-EN, there is a clear difference in speed between the learning of meaningful sentences and random noise. For example, short poems are learned much faster than sequences of random words. In addition, the system initialized with SWOW-EN weights shows greater tolerance to noise.

The issues of AI risk and AI safety are becoming critical as the prospect of artificial general intelligence (AGI) looms larger. The emergence of extremely large and capable generative models has led to alarming predictions and created a stir from boardrooms to legislatures. As a result, AI alignment has emerged as one of the most important areas in AI research. The goal of this position paper is to argue that the currently dominant vision of AGI in the AI and machine learning (AI/ML) community needs to evolve, and that expectations and metrics for its safety must be informed much more by our understanding of the only existing instance of general intelligence, i.e., the intelligence found in animals, and especially in humans. This change in perspective will lead to a more realistic view of the technology, and allow for better policy decisions.

The hippocampus in rodents encodes physical space using place cells that show maximal firing in specific regions of space - their place fields. These place cells are reused across different contexts and environments with uncorrelated place fields. Though place fields are known to depend on distal sensory cues, even identical environments can have completely different place fields if the contexts are different. We propose a novel place cell network model for this feature using two frequently overlooked aspects of neural computation - dendritic morphology and the spatial co-location of spatiotemporally co-active afferent synapses - and show that these enable the reuse of place cells to encode different maps for environments with identical sensory cues.

Text embedding using vector space models has recently emerged as the leading way to represent text in natural language processing. These embeddings can be at the level of words, sentences, or larger textual units including entire documents. However, sentence embeddings are especially useful because sentences are the most explicitly specified elements of individual thoughts or ideas comprising a document, discussion, or conversation. Analyzing text at the sentence level thus allows access to its fine-grained semantics while preserving the semantic structure that is lost in bag-of-words approaches. Several deep learning-based models such as BERT and USE provide such contextual representations. However, the resulting embeddings are very high-dimensional, and the individual dimensions are not amenable to interpretable labels. Thus, these embeddings define a semantic space but not an explicitly useful cognitive map. In this paper, we show that an adaptive clustering approach applied to the embeddings produced by a neural network-based language model can produce much lower-dimensional, readily interpretable semantic representations, thus creating a usable cognitive map for applications such as semantic tracking and visualization of discussions, or discerning the sequential semantic structure of long documents.

Group brainstorming is a common method to generate ideas for open-ended, complex problems. Groups are known to generate fewer and less novel ideas than an equal number of individuals working separately. One cause of this is the tendency of groups to converge prematurely to a small number of ideas and neglect others. It is known that adding structure to brainstorming in the form of external hints tends to increase group performance. Our goal is to develop a cognitively-inspired feedback system that evaluates ideas in real-time and decides what kind of intervention to make in a group brainstorming session, e.g., flag an idea for further exploration, move the conversation in a new direction, etc. In this work, we develop a computational model for detecting relevant, novel, and less elaborated ideas from group messages in real-time. The flagged ideas can be fed back into the group discussion for further elaboration. We explore multiple approaches for relevance detection that do not require out-of-domain data: a probabilistic language model, a zero-shot classifier, and a one-class support vector machine. We compute two types of novelty, (1) absolute novelty with respect to domain knowledge; and (2) relative novelty with respect to previously seen ideas within the group. For absolute novelty, we explore methods to represent the domain semantic knowledge in a compressed way for faster computation. We tested our model on a dataset of ideas generated by over 50 groups. It was found that our probabilistic relevance method is the most accurate and that either topic cluster centers or common domain topics offer similar novelty results compared to full domain representation.

Over the last few decades, modern industrial processes have investigated several cost-effective methodologies to improve the productivity and yield of semiconductor manufacturing. While playing an essential role in facilitating real-time monitoring and control, the data-driven soft-sensors in industries have provided a competitive edge when augmented with deep learning approaches for wafer fault-diagnostics. Despite the success of deep learning methods across various domains, they tend to suffer from bad performance on multi-variate soft-sensing data domains. To mitigate this, we propose a soft-sensing ConFormer (CONvolutional transFORMER) for wafer fault-diagnostic classification task which primarily consists of multi-head convolution modules that reap the benefits of fast and light-weight operations of convolutions, and also the ability to learn the robust representations through multi-head design alike transformers. Another key issue is that traditional learning paradigms tend to suffer from low performance on noisy and highly-imbalanced soft-sensing data. To address this, we augment our soft-sensing ConFormer model with a curriculum learning-based loss function, which effectively learns easy samples in the early phase of training and difficult ones later. To further demonstrate the utility of our proposed architecture, we performed extensive experiments on various toolsets of Seagate Technology’s wafer manufacturing process which are shared openly along with this work. To the best of our knowledge, this is the first time that curriculum learning-based soft-sensing ConFormer architecture has been proposed for soft-sensing data and our results show strong promise for future use in soft-sensing research domain.

Text embedding has recently emerged as a very useful and successful method for semantic representation. Following initial word-level embedding methods such as Latent Semantic Analysis (LSA) and topic-based bag-of-words approaches like Latent Dirichlet Allocation (LDA), the focus has turned to language models and text encoders implemented as neural networks - ranging from word-level models to those embedding whole documents. The distinctive feature of these models is their ability to infer semantic spaces at all levels based purely on data, with no need for complexities such as syntactic analysis or ontology building. Many of these models are available pre-trained on enormous amounts of data, providing downstream applications with general-purpose semantic spaces. In particular, embedding models at the sentence level or higher are most useful in applications because the meaning of text only becomes clear at that level. Most text embedding methods produce text embeddings in high-dimensional spaces, with a dimensionality ranging from a few hundred to thousands. However, it is often useful to visualize semantic spaces in very low dimension, which requires the use of dimensionality reduction methods. It is not clear what language models and what method of dimensionality reduction would work well in these cases. In this paper, we compare four text embedding methods in combination with three methods of dimensionality reduction to map three related real-world datasets comprising textual descriptions of items in a particular domain (sports) to a 2-dimensional semantic visualization space. The results provide several insights into the utility of these methods for data of this type.

The neural basis of spatial cognition and learning in mammals has been studied extensively for several decades. Research has focused in particular on the place cells of the hippocampus and the grid cells found in the entorhinal cortex. In turn, these studies have inspired several models for robotic navigation. One interesting, experimentally observed, feature of spatial learning in rodents is the importance of replay, where animals replay sequences of spatial representations they have experienced in order to learn and make decisions. This feature too has been incorporated into some computational models. In this paper, we describe a new approach to learning navigation in mazes using replay of intrinsically generated sequences rather than relying only on experienced sequences. We show that this improves generalization, and leads to effective one-shot learning that is closer to what is observed in animals.

In creativity research a typical problem is that of assessing the novelty of ideas or solutions generated by many people to open ended problems. For datasets larger than a few hundreds, human assessment of novelty becomes time consuming and error prone. Existing novelty detection methods such as: distance based text similarity or language model approaches do not work well for small datasets. Moreover, when compared to human novelty ratings, these approaches fail to capture the same cognitive processes or biases. We are proposing a novel cognitive model inspired by a leaky accumulator decision making models for detecting novel ideas from short text. The model is applied on a collection of ideas generated in a group brainstorming experiment. It evaluates an idea term by term and it accumulates surprise and relevance. The final novelty decision is taken at the end of each idea by means of a threshold. An important component of the model is a small domain dataset which is used to evaluate the surprise of a term's context compared to common domain knowledge. The model is compared with other methods: feature based classifiers, tf-idf similarity distance, and pretrained language models (ULMFIT).

Politics is an area of broad interest to policy-makers, researchers, and the general public. The recent explosion in the availability of electronic data and advances in data analysis methods - including techniques from machine learning - have led to many studies attempting to extract political insight from this data. Speeches in the U.S. Congress represent an exceptionally rich dataset for this purpose, and these have been analyzed by many researchers using statistical and machine learning methods. In this paper, we analyze House of Representatives floor speeches from the 1981 - 2016 period, with the goal of inferring the partisan affiliation of the speakers from their use of words. Previous studies with sophisticated machine learning models has suggested that this task can be accomplished with an accuracy in the 55 to 80% range, depending on the year. In this paper, we show that, in fact, very comparable results can be obtained using a much simpler linear classifier in word space, indicating that the use of words in partisan ways is not particularly complicated. Our results also confirm that, over the period of study, it has become steadily easier to infer partisan affiliation from political speeches in the United States. Finally, we make some observations about specific terms that Republicans and Democrats have favored over the years in service of partisan expression.

Semantic analysis of text corpora is of broad utility, including for data from conversations, on-line chats, brainstorming sessions, comments on blogs, etc. - all of which are potentially interesting sources of information and ideas. In the present paper, we look at data from a large group brainstorming experiment that generated thousands of mostly brief statements. The ultimate goal is to detect which statements are semantically atypical within the overall corpus. In contexts such as spam detection or detection of on-line intrusions, autoencoders have been used successfully to separate typical from atypical data, and we consider this approach in the present paper. Texts are embedded in a semantic space obtained through topic analysis, and an autoencoder network is used to reconstruct each embedded text. The results show that, while difficulty of reconstruction is related to quantitative measures of atypicality in the embedding vector space, it is not well correlated with novelty assignments made by a human rater. However, this is not the case when the data is first clustered in the embedding space: The reconstruction error for each data cluster indicates that some clusters represent more novel data than others, and that the inverse size of the cluster and the mean reconstruction error of the texts in the cluster capture this well. In particular, autoencoders that enforce dimensionality reduction improve discrimination. The results also show that, in the reconstruction process, the autoencoder implicitly discovers the same clusters in the data that are discovered explicitly by an optimized k-means approach.

Humans often engage in tasks that require or are made more efficient by coordinating with other humans. The coordination involved in these tasks can be understood in terms of the behavioral and affordance dynamics of socially embedded agents engaged in joint action activities. Behavioral dynamics provide mathematical (differential equation) models of human behavior and interaction and affordance dynamics identify and model the ways that an agent's action capabilities evolve over time. Taken together, models of human joint-action based on these approaches may provide a basis for developing robust, natural, and easy to engage artificial agents. In this paper we introduce behavioral and affordance dynamics models of human joint action in a pick-and-place task. Based on these models we provide a proof of concept pick-and-place artificial agent and implement the agent in a 3D virtual environment to interact with human co-actors.

Real-world data such as medical images and sensor measurements is usually high-dimensional and limited. Using such datasets directly in machine learning tasks can lead to poor generalization. Feature learning is a general approach for transforming high-dimensional data points to a representational space with lower dimensionality. Machine learning models can be trained efficiently with such representations. In this paper, a novel feature selection method based on multiple trained sparse auto-encoders (SAEs) is described. It works by selecting diverse, non-redundant features from multiple pinched SAEs with very narrow hidden layers, and then using these features in a more appropriately sized classifier without further feature tuning. The feature learning ability of the method is evaluated in a handwritten digits recognition task. Results show that this type of feature selection provides improved representations for a softmax classifier, and that using pinched SAEs produces results equal to or better than regular SAEs.

Spatiotemporal patterns of neural activity have increasingly come to be seen as important for encoding information in the nervous system, motivating the development of various neurocomputational models. In this paper, we present a simple recurrent neural network model motivated by the need to understand the basis of voluntary motor control. For a given individual, any specific voluntary movement is ultimately encoded as an aperiodic spatiotemporal pattern of activation across a set of muscles, and presumably in spinal and cortical, motor neurons. Over time, such patterns can become stereotypical for the individual, and determine the “style” of specific movements - e.g., how they walk or write an “A”. Experimental studies also indicate that these activity patterns may themselves be constructed as linear combinations of a few fixed spatiotemporal basis patterns of activity called motor synergies. For this to work, it is essential that neural systems be able to represent spatiotemporal activity patterns that are stimulus-specific, aperiodic (i.e., not rhythmic), transient (i.e., lasting only briefly), and robust (i.e., at least somewhat tolerant of errors and noise). The model we describe achieves this by using the dynamics of a recurrent neural network with two classes of primary neurons: Fast neurons that rapidly identify the patterns to be produced based on the stimulus and set up a “scaffolding” for it; and slow neurons that eventually instantiate the relevant spatiotemporal activity pattern. We show that this minimal system exhibits many of the properties needed for the flexible construction of complex, aperiodic movements.

Divergent thinking refers to a style of thinking that ranges across a broad range of concepts, and is considered to be a core enabler of creativity. Thinking is often modeled as a process of conceptual combination, and creative ideas are seen as those using unconventional combinations of concepts. Since conceptual combination is fundamentally an associative process, it has been proposed that creative thinking reflects stronger than expected associations between normally remote associates in the mind of the thinker. We recently proposed a neurodynamical model called Itinerant Dynamics with Emergent Attractors (IDEA) to explore the process by which ideas can emerge in an associative memory system through conceptual combination. In this paper we study how the functional dynamics of this model changes when its associative weights are changed to make remote associations stronger. We apply the model to data from four sets of papers from previous IJCNN meetings. In addition to identifying the differences in the dynamics due to changes in the association pattern, we consider whether divergent thinking can potentially predict future ideas or reconstruct past ideas.

“Happy New Year!” At the beginning of 2016, I would like to take this opportunity to wish everyone a very happy, healthy, and prosperous new year! It is my great honor and privilege to serve as the Editor-in-Chief (EiC) of the IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (TNNLS), and I am excited to write this Editorial to start a new journey with you all.

The availability of unstructured text as a source of data has increased by orders of magnitude in the last few years, triggering extensive research in the automated processing and analysis of electronic texts. An especially important and difficult problem is the identification of salient words in a corpus, so that further processing can focus on these words without distraction by uninformative words. Standard lists of stop words are used to remove common words such as articles, pronouns and prepositions, but many other words that should be removed are much harder to identify because word salience is highly context-dependent. In this paper, we describe a neurodynamical approach for the context-dependent identification of salient words in large text corpora. The method, termed the Attractor Network-based Salient Word Extraction Rule (ANSWER) is modeled as a cognitive mechanism that identifies salient words based on their participation in coherent multi-word ideas. These ideas are, in turn, extracted via attractor dynamics in a recurrent neural network modeling the associative semantic graph of the corpus. The corpus used in this paper comprises the abstracts of all papers published in the proceedings of IJCNN 2009, 2011 and 2013. The list of salient words that the system generates is compared with those generated by other standard metrics, and is found to outperform all of them in almost all cases.

Most models of thought are based on the idea of associative memory: The combination of distinct concepts through their association with each other based on experience. Thus, the pattern of associations in the mind is thought to be a critical factor in determining the ideas - both familiar and unfamiliar - that thought can generate. The Itinerant Dynamics with Emergent Attractors (IDEA) model has been proposed as a computational representation of the process by which ideas might emerge in an associative memory system. However, the effect of how “association” is defined in this context remains an important unexplored issue. In this paper, we consider three different ways of defining associations between concepts based on their joint usage in texts, and show that these lead to very different dynamics in the process of emergent conceptual combinations. We use data from two real-world text corpora to instantiate the model: The collected poems of Dylan Thomas, and the Memoirs of the economist Ludwig von Mises. The two sources differ greatly in the nature of their content as well as the statistics of their word usage, which makes them interesting subjects for comparison.

As the availability of large, digital text corpora increases, so does the need for automatic methods to analyze them and to extract significant information from them. A number of algorithms have been developed for these applications, with topic modeling-based algorithms such as latent Dirichlet allocation (LDA) enjoying much recent popularity. In this paper, we focus on a specific but important problem in text analysis: Identifying coherent lexical combinations that represent "chunks of thought" within the larger discourse. We term these salient semantic chunks (SSCs), and present two complimentary approaches for their extraction. Both these approaches derive from a cognitive rather than purely statistical perspective on the generation of texts. We apply the two algorithms to a corpus of abstracts from IJCNN 2009, and show that both algorithms find meaningful chunks that elucidate the semantic structure of the corpus in complementary ways.

Experimental studies of motor control in humans and other animals suggest that complex movements are constructed from a relatively small set of motor primitives representing preferential coordinated activation patterns in groups of muscles. These have been termed synergies. We have previously presented a neurodynamical model of how motor primitives with the observed characteristics of synergies might be encoded in cortico-spinal and spinal neural networks. The model showed that a small basis set of synergies could be used to combinatorially generate linear trajectories in all directions from all points within the posture space of a two-joint, two degree-of-freedom arm. We now present an extension of that model, where useful combinations of these low-level synergies are encoded into higher-level primitives termed hypersynergies, such that the activation of a single hypersynergy with appropriate control parameters allows the generation of an extensive repertoire of movements over large parts of posture space. This repertoire is “exploited” by a cortical motor control system implemented through interacting neural maps. We argue that this system can generate complex movements with relatively simple neural control mechanisms.

In this paper we study the effect of external ideas on brainstorming by means of two computational models: a transient emergent attractors model (TEAM) and a probabilistic associative model (PAM). New behavioral experimental results show that hints or others' ideas can either hinder or enhance ideas generated during exposure period, while they consistently enhance the quantity of ideas produced after exposure. The TEAM model consists of a neural network of concept nodes connected by means of category membership and relatedness. Ideas emerge dynamically from the activity of the network by temporarily strengthening the connections between co-active nodes. Local inhibition inactivates current idea nodes and allows another idea to form. Active nodes prime connected inactive nodes depending on the recent activity of the node. The model shows that hindering of the number of ideas during hint presentation depends on the strength of hints and that the speed and duration of priming is essential for the long-term priming effect of hints observed in experiments. For comparison purposes, a PAM model originally proposed by Brown and Paulus (1998) is used to explain the same experimental data.

The neural basis of creative thinking - indeed of all thinking - remains mysterious. One influential theory by Mednick holds that creative thinking reflects a difference in the associational structure of conceptual representations in the mind. We have previously proposed a neural network model based on itinerant dynamics to model thinking, and used it to show that a small-world, scale-free associational structure - similar to that found empirically in linguistic data - is especially efficient for exploring conceptual space and generating conceptual combinations. In this paper, we apply this model to associative networks obtained from the poetry of Dylan Thomas and John Gay, and the prose of F. Scott Fitzgerald and George Orwell. Network analysis shows that poetic texts indeed incorporate a wider distribution of associations than prose. However, neural simulations using semantic networks from the four sources present a more complex picture. We also consider the case where a poet's associative network is transformed to that of a prose-writer to test the impact of this manipulation.

Understanding the ability of humans and animals to exhibit a large repretoire of complex movements in a continuosly changing and uncertain environment is of interest to both biologists and engineers. Even the simplest movements require complex control of internal and external variables of the body and the environment in a variety of contexts. Classical methods - such as those used in industrial robotics - are difficult to apply in these high degree-of-freedom situations. Studies on motor control in animals have led to the discovery that, rather than using standard feedback control based on continuous tracking of desired trajectories, animals' movements emerge from the controlled combination of pre-configured movement primitives or synergies. These synergies define coordinated patterns of activity across specific sets of muscles, and can be triggered as a whole with controlled amplitude and temporal offset. Combinations of synergies, therefore, allow emergent configuration of a wide range of complex movements. Control is both simpler and richer in this synergistic framework because it is based on selection and combination of synergies rather than myopic tracking of trajectories. Though the existence of motor synergies is now well-established, there is very little computational modeling of them at the neural level. In this paper, we describe a simple neural model for motor synergies, and show how a small set of synergies selected through a redundancy-reduction principle can generate a rich motor repertoire in a model two-jointed arm system.

The amount of news delivered by the different media in the current environment can be overwhelming. Although the events being reported are factually the same, the ways with which the news is delivered vary with the media sources involved. In many cases, it is difficult to reliably uncover the latent information hidden within the news reports due to the great diversity of topics and the sheer volume of news. Analysis of the news media has always been of interest to news analysts, politicians and policy makers in order to aggregate and make sense of the information generated every day. News sources try to achieve relevance to their audiences by providing them with news that the audience wants or finds interesting, but often also have implicit motives such as shaping the perceptions of their audience. Although these agendas or target audiences are not explicitly identified, we consider ways in which this information can be inferred by applying the tools of natural language processing and semantic analysis to the news streams from these sources.

The abstraction of patterns from data and the formation of categories is a hallmark of human cognitive ability. As such, it has been studied from many different perspectives by researchers, and these studies have led to several explanatory models. In this paper, we consider the inference of categorical representations for the purpose of producing task-specific responses. Task-relevant responses require a knowledge repertoire that is organized to allow efficient access to useful information. We present a neurodynamical system that infers functionally coherent categories from semantic inputs (or concepts) presented sequentially in different contexts, and encodes them as attractors in a two-dimensional topological feature space. The resulting category representations can then act as pointers in a larger system for semantic cognition. The system allows controlled hierarchical organization and functional segregation of the inferred categories.

Creativity and insight are distinctive attributes of human cognition, but their neural basis remains poorly understood due to the difficulty of experimental study. As such, computational modeling can play an important role in understanding these phenomena. Some researchers have proposed that creative individuals have a “deeper” organization of knowledge, allowing them to connect remote associates and form novel ideas. It is reasonable to assume that the depth and richness of semantic organization in individual minds is related to the connectivity of neural networks involved in semantic representation. In this paper, we use a simple and plausible neurodynamical model of semantic networks to study how the connectivity structure of these networks relates to the richness of the semantic constructs, or ideas, they can generate. This work is motivated, in part, by research showing that experimentally obtained semantic networks have a specific connectivity pattern that is both small-world and scale-free. We show that neural semantic networks reflecting this structure have richer semantic dynamics than those with other connectivity structures. Though simple, this model may provide insight into the important issue of how the physical structure of the brain determines one of the most profound features of the human mind - its capacity for creative thought.

The self-organization of social networks and the emergence of ideas have both been studied extensively in recent years, but seldom in a single framework. In this paper, we describe a distributed multi-agent model for the self-organization of social networks from encounters between agents with specific ideas, which are seen as combinations of words. Each agent maintains a semantic network of the words it knows, which implicitly defines the ideas in its repertoire. Agents exchange their ideas over their social networks, and incorporate the received ideas in their semantic networks. Social bonds are made and broken based on the agents' social and semantic preferences (i.e., shared ideas), leading to the emergence of social communities. Thus, the model embodies a circular interaction between the formation of social networks and new ideas. We mine the resulting communities for novel ideas that are generated by their members, and look at the effect of interaction choices on their formation.

Creativity is an important attribute of the human mind, and shows itself in all aspects of its function. However, its neural basis remains poorly understood. In this paper, we explore two issues with regard to creativity in the semantic domain: 1) What neural mechanism enable the brain to construct context-specific semantic spaces to facilitate the generation of relevant ideas? and 2) Can these mechanisms support greater creativity simply by exploring unusual semantic spaces? We use a variant of our previously developed neural model of ideation to show that a dynamical modular neural system can, indeed, learn to configure context-appropriate semantic domains based on experience, and that exploratory dynamics within this system can lead to the unmasking of novel emergent ideas.

As advances in neuroscience, cognitive science and robotics continue to elucidate the physical basis of autonomous behavior in animals, many deep questions remain open, including: What sort of system can evolve and support perception, cognition and action? What general principles or attributes can be defined for such a system? If so, how do these principles and attributes make complex behavior possible? In this paper, we report on the early stages of research to address these questions from a complex systems and systems biology perspective. We argue that flexible modularity and hierarchical interaction of synergies in a modular system are fundamental to the possibility of cognition. We describe a minimal model comprising three interacting core subsystems that embodies the principles of flexible modularity and synergy. Finally, we present a simple neural model of action generation based on the principles of interacting modular synergies.

Understanding the neural mechanisms of the idea generation process has implications for research in brainstorming, creativity and innovation. In this paper we present a conceptual neural model for generating ideas. The model extends the associative memory model of Brown et al. (1998) by explicitly representing categories as networks of concepts and ideas as conceptual combinations. Simulation results are compared with experimental results on effects of priming on low, versus high accessibility categories.

Brainstorming is the process of generating ideas in a specific task or problem context.We have previously presented a connectionist framework to study the dynamics of idea generation in individuals. In this paper, we develop this model further, and apply it to studying qualitatively the effects of priming on the process of ideation. Motivated by experimental data from a previous study, we explore the differential effects of relevant and irrelevant primes on productivity of idea generation in specific problem/task contexts. Simulations using our model suggest that even irrelevant primes can provide a modest productivity boost in contexts that are familiar or are similar to familiar contexts, but no benefit when the context is unfamiliar. We propose possible explanations for these results and make predictions for future experiments.

In this paper, we present a model for the generation of ideas within a creative thinking/brainstorming context. In the model, ideas emerge as conceptual combinations from the interaction of complex dynamics at several semantic levels: features, concepts, categories, and previously generated ideas. This dynamics is shaped by external information on task context, constraints and goals, and is modulated by evaluative feedback from an internal critic working through reinforcement. While the model is abstract, it attempts to capture the interplay between semantic representations in the temporal, frontal and parietal cortices, working memory in the prefrontal cortex, attentional selection by the basal ganglia, and modulation from the dopaminergic reward system. We show that a context-specific itinerant search for novel but meaningful conceptual combinations (ideas) emerges naturally from the dynamics of this system. We also briefly describe a computational model for ideation in groups using a multi-agent formalism. The initial focus of this model is on studying the potential benefits of cognitive diversity in agent groups, e.g., the presence of convergent and divergent thinkers, or agents with different semantic organizations.

Recurrent neural networks functioning as associative memories are often studied and optimized for recall quality and capacity, with the focus primarily on the network's stability, i.e., convergence to stored attractors. However, the ability of networks to switch between attractors in a controlled way is also potentially a useful phenomenon. Networks that are stable under most conditions, but can be switched by specific stimuli may be used to model cognitive control and other timevarying cognitive phenomena. Such networks, which we term stable-yet-switchable (SyS) networks, are also of interest from the networks perspective, and the SyS properties of scale-free networks have been noted by researchers. In this paper, we consider networks with bimodal connectivity - a core of densely connected neurons and a larger periphery with sparser connectivity - and compare their SyS performance with random and scale-free recurrent neural networks. The results show that core-periphery networks have much better SyS performance than scale-free networks.

Cognitive control -the ability to produce appropriate behavior in complex situations -is a fundamental aspect of intelligence. It is increasingly evident that this control arises from the interaction of dynamics in several brain regions, and depends significantly on processes of modulation and dynamical biasing. While most research has focused on explanations of behavioral responses seen in experiments and pathologies, it is reasonable to expect that internal functions such as planning and thinking would also use similar control mechanisms. In this paper, we present a connectionist model for an idea generation process that can rapidly retrieve old ideas in familiar contexts and search for novel ideas in unfamiliar ones. Based on a simple reinforcement signal, the system learns context-dependent biases that represent effective internal "response systems" for generating ideas from conceptual elements. A broad goal of the research is to show that preconfigured structural modularity, limited real-time selectivity, and adaptive modulation can interact to produce the flexible functionality necessary for cognition and intelligent behavior.

Ad hoc wireless sensor networks are emerging as an important technology for applications such as environmental monitoring, battlefield surveillance and infrastructure security. While most research so far has focused on the network aspects of these systems (e.g., routing, scheduling, etc.), the capacity for scalable, in-field information processing is potentially their most important attribute. Networks that can infer the phenomenological structure of their environment can use this knowledge to improve both their sensing performance and their resource usage. These intelligent networks would require much less a priori design, and be truly autonomous. This paper presents a distributed algorithm for inferring the global topological connectivity of an environment through a simple self-organization algorithm based on Hebbian learning. The application considers sensors distributed over an environment with a network of tracks on which vehicles of various types move according to rules unknown to the sensor network. Each sensor infers the local topology of the track network by comparing its observations with those from neighboring sensors. The complete topology of the network emerges from the distributed fusion of these local views.

Ad-hoc sensor networks comprising large numbers of randomly deployed wireless sensors have recently been an active focus of investigation. These networks require self-organized configuration after deployment, and ad-hoc heuristic methods for such configuration have been proposed with regard to many aspects of the networks' performance. However, systematic approaches for such configuration remain elusive. In this paper, we present a preliminary attempt towards such a systematic approach using evolutionary algorithms and reverse engineering. In particular, we focus on the problem of obtaining heterogeneous networks that optimize global functional properties through local adaptive rules. Almost all work on ad-hoc sensor network has so far involved homogeneous networks where all nodes transmit with the same power level, creating a symmetric connectivity. It is possible to construct heterogeneous networks by allowing nodes to transmit at different power levels, and such networks are known to provide improvements in network lifetime, power efficiency, routing, etc. However, such networks are difficult to build mainly because the optimal power level for each node depends on the node location and spatial context, which are not known before deployment. A few heuristic schemes focused on improving power consumption have been proposed in the literature, but the issue has not been investigated sufficiently at a general level. In this paper, we present a new and improved heuristic developed using a reverse engineered approach. A genetic algorithm is used to generate a set of heterogeneous sensor networks that are characterized by low short paths and minimal congestion. Analysis of this optimal network set yields rules that form the basis for a local heuristic. We show that networks adapted using this heuristic produce significant improvement over the homogeneous case. More importantly, the results validate the utility of the proposed approach that can be used in other self-organizing systems.

While most existing research efforts in the area of wireless sensor networks have focused on networks with identical nodes, deploying sensors with different capabilities has become a feasible choice. In this paper, we focus on sensor networks with two types of nodes that differ in their capabilities, and discuss the effects of heterogeneity of sensing and transmission ranges on the network coverage and broadcast reachability. Our work characterizes how the introduction of a few sensor nodes with better capabilities can reduce the number of total required sensors without sacrificing the coverage and the broadcast reachability. Analytical results are validated via simulations. This work can serve as a guideline for designing large-scale sensor networks cost-effectively. It can also be extended to more complicated heterogeneous wireless sensor networks with more than two types of sensors.

This paper considers a heterogeneous team of cooperating unmanned aerial vehicles (UAVs) drawn from several distinct classes and engaged in a search and action mission over a spatially extended battlefield with targets of several types. During the mission, the UAVs seek to confirm and verifiably destroy suspected targets and discover, confirm, and verifiably destroy unknown targets. The locations of some (or all) targets are unknown a priori, requiring them to be located using cooperative search. In addition, the tasks to be performed at each target location by the team of cooperative UAVs need to be coordinated. The tasks must, therefore, be allocated to UAVs in real time as they arise, while ensuring that appropriate vehicles are assigned to each task. Each class of UAVs has its own sensing and attack capabilities, so the need for appropriate assignment is paramount. In this paper, an extensive dynamic model that captures the stochastic nature of the cooperative search and task assignment problems is developed, and algorithms for achieving a high level of performance are designed. The paper focuses on investigating the value of predictive task assignment as a function of the number of unknown targets and number of UAVs. In particular, it is shown that there is a tradeoff between search and task response in the context of prediction. Based on the results, a hybrid algorithm for switching the use of prediction is proposed, which balances the search and task response. The performance of the proposed algorithms is evaluated through Monte Carlo simulations.

The paper presents a neural model for learning sequences of relevant patterns embedded in distractors. A contextual episode is a sequence of relevant patterns - always in the same order - intermixed with distractors. By repeated presentations of all contextual episodes, the model discovers for each episode the set of relevant patterns and their order. The problem is solved in two stages: (a) by eliminating distractors, and (b) by learning the order between relevant patterns. The model uses the concept of latent attractors - essential in creating different neural representations for same patterns in distinct episodes. No external teacher and only Hebbian type learning rules are used.

Most algorithms for motion-based segmentation depend on the system's ability to estimate optic flow from successive image frames. Block-matching is often used for this, but it faces the problems of noise-sensitivity and texture-insufficiency. Recently, we proposed a two-pathway approach based on locally coupled neural networks to address this issue. The system uses a pixel-level (P) pathway to perform robust block-matching in regions with sufficient texture, and a region-level (R) pathway to estimate motion from feature matching in low-texture regions. The fused optic-flow from the P and R pathways is then segmented by a pulse-coupled neural network (PCNN). The algorithm has produced very good results on synthetic and natural images. We show that its performance shows significant robustness to additive noise in the images.

Motion-based segmentation is a very important capability for computer vision and video analysis. It depends fundamentally on the system's ability to estimate optic flow using temporally proximate image frames. This is often done using block-matching. However, block-matching is sensitive to the presence of observational noise, which is inevitable in real images. Also, images often include regions of homogeneous intensity, where block-matching is problematic. A better method in this case is to estimate motion at the region level. In the approach described in this paper, we have attempted to address the noise-sensitivity and texture-insufficiency problems using a two-pathway system. The pixel-level pathway is a multilayer pulse-coupled neural network (PCNN)-like locally coupled network used to correct outliers in the block-matching motion estimates and produce improved estimates in regions with sufficient texture. In contrast, the region-level pathway is used to estimate the motion for regions with little intensity variation. In this pathway, a PCNN network first partitions intensity images into homogeneous regions, and a motion vector is then determined for the whole region. The optic flows from both pathways are fused together based on the estimated intensity variation. The fused optic flow is then segmented by a one-layer PCNN network. Results on synthetic and real images are presented to demonstrate that the accuracy of segmentation is improved significantly by taking advantage of the complementary strengths and weaknesses of the two pathways.

Latent attractor networks have been proposed as a possible mechanism for representing episodic context in the hippocampus, and as general purpose models of episodic context-dependent encoding in neural networks. These are recurrent neural networks with attractors that never fully manifest themselves, but bias the network's response to external stimuli. While each attractor in the original latent attractor model was triggered by unique context patterns specific to the context, this model was later extended to the case where contexts were triggered progressively by the sequential presentation of several stimulus patterns without regard to order, simulating the more realistic situation where a context is identified by a sequentially scanned combination of landmarks. In this paper, we describe a network model that can select among contexts identified by overlapping sequences of different lengths, even if the relevant stimulus patterns are interspersed among patterns irrelevant to context selection.

"Small-World" networks is a term recently coined by Watts and Strogatz to describe networks which simultaneously exhibit a high degree of node clustering and short minimum path lengths between nodes. Such networks represent a very efficient architecture for achieving maximal internode communication with minimal connection length-a feature that is extremely important in highly connected physical networks, where interconnections consume most space. Neural networks-both in the brain and in hardware implementation-can benefit greatly from a small-world architecture, and there is evidence that this strategy is widely used in the nervous system. In this paper, we study the recall performance of associative memories with regard to their small-world characteristics. The results indicate that, indeed, a small-world approach can lead to networks with high performance and minimal interconnect requirements.

We (1999) have proposed a paradigm called "latent attractors" where attractors embedded in a recurrent network via Hebbian learning are used to channel network response to external input rather than becoming manifest themselves. This allows the network to generate context-sensitive internal codes in complex situations. Latent attractors are particularly helpful in explaining computations within the hippocampus-a brain region of fundamental significance for memory and spatial learning. The performance of latent attractor networks depends on the number of such attractors that a network can sustain. Following methods developed for associative memory networks, we present analytical and computational results on the capacity of latent attractor networks.

Cells throughout the rodent hippocampal system show place-specific patterns of firing called place fields, creating a coarse-coded representation of location. The dependencies of this place code--or cognitive map--on sensory cues have been investigated extensively, and several computational models have been developed to explain them. However, place representations also exhibit strong dependence on spatial and behavioral context, and identical sensory environments can produce very different place codes in different situations. Several recent studies have proposed models for the computational basis of this phenomenon, but it is still not completely understood. In this article, we present a very simple connectionist model for producing context-dependent place representations in the hippocampus. We propose that context dependence arises in the dentate gyrus-hilus (DGH) system, which functions as a dynamic selector, disposing a small group of granule and pyramidal cells to fire in response to afferent stimulus while depressing the rest. It is hypothesized that the DGH system dynamics has "latent attractors," which are unmasked by the afferent input and channel system activity into subpopulations of cells in the DG, CA3, and other hippocampal regions as observed experimentally. The proposed model shows that a minimally structured hippocampus-like system can robustly produce context-dependent place codes with realistic attributes.

In many cognitive situations, context is specified globally-encompassing the entire duration of an episode, but explicitly identified only at its initiation. Examples of such contexts are social situations, spatial environments, task specifications, etc. In many such cases, a neural system needs to respond differently to the same immediate stimulus depending on the global context. Clearly, the system must be able to switch into different response regimes and remain there while the context remains fixed. Simultaneously, it must remain sensitive to external stimuli, and respond to them in a reliable and informative manner. We propose a neurally plausible approach for handling these requirements.

Covariance learning is a powerful type of Hebbian learning, allowing both potentiation and depression of synaptic strength. It is used for associative memory in feedforward and recurrent neural network paradigms. This article describes a variant of covariance learning that works particularly well for correlated stimuli in feedforward networks with competitive K-of-N firing. The rule, which is nonlinear, has an intuitive mathematical interpretation, and simulations presented in this article demonstrate its utility.

We investigate the dynamics of a class of recurrent random networks with sparse, asymmetric excitatory connectivity and global shunting inhibition mediated by a single interneuron. Using probabilistic arguments and a hyperbolic tangent approximation to the gaussian, we develop a simple method for setting the average level of firing activity in these networks. We demonstrate through simulations that our technique works well and extends to networks with more complicated inhibitory schemes. We are interested primarily in the CA3 region of the mammalian hippocampus, and the random networks investigated here are seen as modeling the a priori dynamics of activity in this region. In the presence of external stimuli, a suitable synaptic modification rule could shape this dynamics to perform temporal information processing tasks such as sequence completion and prediction.

An investigation is presented of an extension, proposed by A.A. Minai and R.D. Williams (Proc. Int. Joint Conf. on Neural Networks, vol.1, p.676-79, Washington, DC, 1990), to an algorithm for training neural networks in real-valued, continuous approximation domains. Specifically, the most effective aspects of the proposed extension are isolated. It is found that while momentum is particularly useful for the delta-bar-delta algorithm, it cannot be used conveniently because of sensitivity considerations. It is also demonstrated that by using more subtle versions of the algorithm, the advantages of momentum can be retained without any significant drawbacks

The significant computational requirements of VLSI layout suggest that it may be desirable to estimate the feasibility of a task before actually performing the task. The system described in this paper uses a discrete heuristics approach to estimate the future quality of a semi-custom layout before any placement or routing is done. It does this evaluation with respect to critical parameters such as routability, area utilization, and wire length, using heuristics arranged in a discrete graph structure. The system can handle user-specified non-rectangular layout shapes. Its rule-based structure allows easy observation and modification of individual heuristics for the purposes of “fine tuning.” The system also detects potential problems and suggests possible solutions.