Memory is a key functional requirement for cognitive agents. There are three basic ways to implement memory using neural networks: (1) RNN: recurrent neural networks, (2) TDNN: time-delayed neural networks (feed-forward), and (3) DROPPER: external marker dropper/detector (feed-forward). All three have been found to be effective in prior research. In this paper, we ask which of these mechanisms could have evolved earlier/easier? To answer this question, we set up a simple ball-catching task where two balls fall from above at different speeds, and an agent at the bottom has to catch the balls using range sensors. Depending on the relative speed of the balls, sometimes the slow ball will go out of sensor range, thus to catch the fast ball first then remember to catch the second (slow) ball, memory is required. We used the Neuroevolution of Augmenting Topologies (NEAT) algorithm to evolve all three types of memory mechanisms, where not only the connection weights but also the network topologies are evolved. Our results show that the DROPPER mechanism is the fastest to evolve a successful controller, followed by TDNN and RNN. Among the feed-forward topologies, we also found that DROPPER is more robust than TDNN (less sensitive to the relative speed of the balls). These results show that a simple reactive agent could quickly evolve a rudimentary form of memory through depositing and detecting external markers, long before other internalized memory mechanisms evolve. These findings shed light on the evolutionary route toward memory in cognitive agents.

Vision and proprioception have fundamental sensory mismatches in delivering locational information, and such mismatches are critical factors limiting the efficacy of motor learning. However, it is still not clear how and to what extent this mismatch limits motor learning outcomes. To further the understanding of the effect of sensory mismatch on motor learning outcomes, a reinforcement learning algorithm and the simplified biomechanical elbow joint model were employed to mimic the motor learning process in a computational environment. By applying a reinforcement learning algorithm to the motor learning of elbow joint flexion task, simulation results successfully explained how visual-proprioceptive mismatch limits motor learning outcomes in terms of motor control accuracy and task completion speed. The larger the perceived angular offset between the two sensory modalities, the lower the motor control accuracy. Also, the more similar the peak reward amplitude of the two sensory modalities, the lower the motor control accuracy. In addition, simulation results suggest that insufficient exploration rate limits task completion speed, and excessive exploration rate limits motor control accuracy. Such a speed-accuracy trade-off shows that a moderate exploration rate could serve as another important factor in motor learning.

How individuals' movement decisions lead to complex, large-scale behavior of crowds is a central subject in crowd modeling and simulation. Understanding this multiscale, emergent phenomenon can provide deeper insights into social dynamics in a crowd, and can help design real world applications such as evacuation planning systems. Crowd simulation is generally computationally intensive, especially when complex motion planning constraints are imposed on individual actors. Furthermore, altering such constraints requires major changes in the simulation engine. An effective way to deal with these issues is the use of intangible social factors (proxies). Proxies are agent-like entities that are dynamically generated and destroyed. They can be combined with a generic, domain-independent motion planner for computationally efficient simulation. These proxies are generally generated and destroyed based on a fixed set of rules, mimicking intangible social factors among individuals. In this paper, we propose the use of evolving neural networks (Neuroevolution of Augmenting Topologies, NEAT) to dynamically control the use of proxies. Our results show that the neural networks evolve to dynamically utilize the proxies, and proxy use increases performance in a simple evacuation task. These results suggest that proxy rules can be learned based on loosely defined fitness goals. We expect our method to be applicable to more complex behavioral modeling and simulation domains, beyond crowd modeling.

Multiscale design is the widely practiced use of space and scale to visually explore and articulate relationships. Free-form web curation (FFWC) is an approach to supporting multiscale design, involving creative strategies of collecting content, assembling it to juxtapose and organize, sketching, writing, shifting perspective to navigate, and exhibiting to share and collaborate. Our long term goal is to support design students with automatic, on demand feedback.

Tool use is an important milestone in the evolution of intelligence. In this paper, we investigate different modes of tool use that emerge in a reaching and dragging task. In this task, a jointed arm with a gripper must grab a tool (T, I, or L-shaped) and drag an object down to the target location (the bottom of the arena). The simulated environment had real physics such as gravity and friction. We trained a deep-reinforcement learning based controller (with raw visual and proprioceptive input) with minimal reward shaping information to tackle this task. We observed the emergence of a wide range of unexpected behaviors, not directly encoded in the motor primitives or reward functions. Examples include hitting the object to the target location, correcting error of initial contact, throwing the tool toward the object, as well as normal expected behavior such as wide sweep. Also, we further analyzed these behaviors based on the type of tool and the initial position of the target object. Our results show a rich repertoire of behaviors, beyond the basic built-in mechanisms of the deep reinforcement learning method we used.

Deep Reinforcement Learning (RL) algorithms are widely being used in autonomous driving due to their ability to cope with unseen environments. However, in a complex domain like autonomous driving, these algorithms need to explore the environment enough to be able to converge. Therefore, these algorithms are faced with the problem of long training times and large amounts of data. In addition, using deep RL algorithms in areas that safety is an important factor such as autonomous driving can lead to a safety issue since we cannot leave the car driving in the street unattended. In this research, we tested two methods for the purpose of reducing the training time. First, we pre-trained Soft Actor-Critic (SAC) with Learning from Demonstrations (LfD) to find out if pre-training can reduce the training time of the SAC algorithm. Then, an online end-to-end combination method of SAC, LfD, and Learning from Interventions (LfI) is proposed to train an agent (dubbed Online Virtual Training). Both scenarios were implemented and tested in an inverted-pendulum task in OpenAI gym and autonomous driving in the Carla simulator. The results showed a dramatic reduction in the training time and a significant increase in gaining rewards for Online LfD (33%) and Online Virtual training (36 %) as compare to the baseline SAC. The proposed approach is expected to be effective in daily commute scenarios for autonomous driving.

Architecture optimization, which is a technique for finding an efficient neural network that meets certain requirements, generally reduces to a set of multiple-choice selection problems among alternative sub-structures or parameters. The discrete nature of the selection problem, however, makes this optimization difficult. To tackle this problem we introduce a novel concept of a trainable gate function. The trainable gate function, which confers a differentiable property to discrete-valued variables, allows us to directly optimize loss functions that include non-differentiable discrete values such as 0-1 selection. The proposed trainable gate can be applied to pruning. Pruning can be carried out simply by appending the proposed trainable gate functions to each intermediate output tensor followed by fine-tuning the overall model, using any gradient-based training methods. So the proposed method can jointly optimize the selection of the pruned channels while fine-tuning the weights of the pruned model at the same time. Our experimental results demonstrate that the proposed method efficiently optimizes arbitrary neural networks in various tasks such as image classification, style transfer, optical flow estimation, and neural machine translation.

Consider a natural language sentence describing a specific step in a food recipe. In such instructions, recognizing actions (such as press, bake, etc.) and the resulting changes in the state of the ingredients (shape molded, custard cooked, temperature hot, etc.) is a challenging task. One way to cope with this challenge is to explicitly model a simulator module that applies actions to entities and predicts the resulting outcome (Bosselut et al. 2018). However, such a model can be unnecessarily complex. In this paper, we propose a simplified neural network model that separates action recognition and state change prediction, while coupling the two through a novel loss function. This allows learning to indirectly influence each other. Our model, although simpler, achieves higher state change prediction performance (67% average accuracy for ours vs. 55% in (Bosselut et al. 2018)) and takes fewer samples to train (10K ours vs. 65K+ by (Bosselut et al. 2018)).

Estimating the relative importance of each sample in a training set has important practical and theoretical value, such as in importance sampling or curriculum learning. This kind of focus on individual samples invokes the concept of samplewise learnability: How easy is it to correctly learn each sample (cf. PAC learnability)? In this paper, we approach the sample-wise learnability problem within a deep learning context. We propose a measure of the learnability of a sample with a given deep neural network (DNN) model. The basic idea is to train the given model on the training set, and for each sample, aggregate the hits and misses over the entire training epochs. Our experiments show that the samplewise learnability measure collected this way is highly linearly correlated across different DNN models (ResNet-20, VGG-16, and MobileNet), suggesting that such a measure can provide deep general insights on the data’s properties. We expect our method to help develop better curricula for training, and help us better understand the data itself.

Long Short-Term Memory (LSTM) has been proven an efficient way to model sequential data, because of its ability to overcome the gradient diminishing problem during training. However, due to the limited memory capacity in LSTM cells, LSTM is weak in capturing long-time dependency in sequential data. To address this challenge, we propose an Attention-aware Bidirectional Multi-residual Recurrent Neural Network (ABMRNN) to overcome the deficiency. Our model considers both past and future information at every time step with omniscient attention based on LSTM. In addition to that, the multi-residual mechanism has been leveraged in our model which aims to model the relationship between current time step with further distant time steps instead of a just previous time step. The results of experiments show that our model achieves state-of-the-art performance in classification tasks.

End-to-end learning in deep reinforcement learning based on raw visual input has shown great promise in various tasks involving sensorimotor control. However, complex tasks such as tool use require recognition of affordance and a series of non-trivial subtasks such as reaching the tool, grasping the tool, and wielding the tool. In such tasks, end-to-end approaches with only the raw input (e.g. pixel-wise images) may fail to learn to perform the task or may take too long to converge. In this paper, inspired by the biological sensorimotor system, we explore the use of proprioceptive/kinesthetic inputs (internal inputs for body position and motion) as well as raw visual inputs (exteroception, external perception) for use in affordance learning for tool use tasks. We set up a reaching task in a simulated physics environment (MuJoCo), where the agent has to pick up a T-shaped tool to reach and drag a target object to a designated region in the environment. We used an Actor-Critic-based reinforcement learning algorithm called ACKTR (Actor-Critic using Kronecker-Factored Trust Region) and trained it using various input conditions to assess the utility of proprioceptive/kinesthetic inputs. Our results show that the inclusion of proprioceptive/kinesthetic inputs (position and velocity of the limb) greatly enhances the performance of the agent: higher success rate, and faster convergence to the solution. The lesson we learned is the important factor of the intertwined relationship of exteroceptive and proprioceptive in sensorimotor learning and that although end-to-end learning based on raw input may be appealing, separating the exteroceptive and proprioceptive/kinesthetic factors in the input to the learner, and providing the necessary internal inputs can lead to faster, more effective learning.

Unlike previous unknown nouns tagging task, this is the first attempt to focus on out-of-vocabulary (OOV) lexical evaluation tasks that does not require any prior knowledge. The OOV words are words that only appear in test samples. The goal of tasks is to provide solutions for OOV lexical classification and predication. The tasks require annotators to conclude the attributes of the OOV words based on their related contexts. Then, we utilize unsupervised word embedding methods such as Word2Vec and Word2GM to perform the baseline experiments on the categorical classification task and OOV words attribute prediction tasks.

Time delays are universal in an organism's nervous system. A majority of them are the results of the limited propagation speed of action potential through the axons. They are inevitable and commonly considered as obstacles to overcome. However, many studies have shown that delays in the nervous system have a nonuniform distribution which helps stabilize the dynamics of the network, leads to greatly increased information capacity, and enable the emergence of the brains predictive function. Additionally, our previous work indicates that the brains predictive function may utilize facilitating neuronal dynamics to generate short-term plasticity (decrease or increase in synaptic transmission) for delay compensation purposes. In this study, we demonstrate how adaptive synaptic delay, together with facilitating neuronal dynamics, can be used to build a sensorimotor controller for a dynamic control task by utilizing simple feedforward neural networks, all under impoverished and long delayed input conditions. Our findings confirm that through adaptive delay and facilitating neuronal dynamics, feedforward neural networks develop a strong memory-like mechanism and exhibit rich dynamic behaviors, successfully solving a tough dynamic control task. We expect our results to shed new light on the role of adaptive synaptic delay and facilitating dynamics in the nervous system, in relation to memory-like mechanisms.

Tool construction requires sophisticated cognitive function and is only observed in higher mammals and a few avian species. In this paper, we will examine the spontaneous emergence of tool construction during the simulated evolution of a two-degree-of-freedom articulated limb controller in a reaching task environment. The limb controller is a recurrent neural network with a topology evolved using the NeuroEvolution of Augmenting Topologies (NEAT) algorithm. First, we show how broad fitness criteria such as distance to target, number of successful reaches, number of steps to reach the target, and number of instances holding the correct length tool are enough to give rise to tool construction. Second, we analyze how the number of tools and their location in the environment during evolution affect the evolved neural circuits' ability to detect tool affordances and employ the optimal decision strategy. We expect our results to help us understand the implications of tool use capability and the environmental conditions that may facilitate its development.

Cloud computing has seen rapid growth due to its massive scalability in storage and computing power. Leading the trend, IBM released a hybrid cloud development platform, based on infrastructure as a service. Although tens of thousands of customers visit the platform everyday, a large percentage of trial customers left as their free-trial access expired, and a high proportion of paying customers dropped their usage sharply or worse stopped their payment. Customer retention in marketing is critical for reduced cost in retaining temporary customers and higher profits from long-term customers. In this paper, we present a data-driven iterative churn prediction framework with a deep learning approach for everything as a service (XaaS) in the cloud, including a cloud platform or software. Moreover, to improve churn prediction analysis we propose a new temporal customer engagement analysis model, called Nascency, Intermediate, and Latest (NIL) analysis. The NIL analysis helps to capture the temporal changes from time-related usage features, obtaining the three standardized inputs from instances which have different lifetimes. To the best of our knowledge this is the first study to use deep learning for churn prediction with time-correlation features in cloud computing. Our experiments have revealed strengths and weaknesses of deep learning in churn prediction. We expect these insights to help us improve deep learning's performance in churn prediction, by exploiting its hierarchical abstraction capability.

A cloud platform website, offering a catalog of services, operates under a freemium business model or a free trial business model, aggressively marketing to customers who have previously visited. In such a cloud platform or service business, accurate identification of high profile customers is central to the success for the business. However, there are several limitations of existing approaches because of the following challenges: (1) heavy customer traffic flows, (2) the noise in user behaviors, (3) a lack of collaboration across stakeholders, (4) class imbalanced customer data (few paying customers vs. high numbers of freemium or trial customers), and (5) unpredictable business environments. In this paper, we propose a data-driven iterative sales lead prediction framework for cloud everything as a service (XaaS), including a cloud platform or software. In this framework, from the BizDevOps process we collaborate to extract business insights from multiple business stakeholders. From these business insights, we calculate service usage scores using our RFDL (Recency, Frequency, Duration, and Lifetime) analysis and estimate sales lead prediction based on the usage scores in a supervised manner. Our framework adapts to a continuously changing environment through iterations of the whole process, maintains its performance of sales lead prediction, and finally shares the prediction results to the sales or marketing team effectively. A three-month pilot implementation of the framework led to more than 300 paying customers and more than $200K increase in revenue. We expect our scalable, iterative sales lead prediction approach to be widely applicable to online or cloud business domains where there is a constant flux of customer traffic.

Tool use constitutes a range of complex behaviors that generally require a sophisticated level of cognition, and is only found in higher mammals and a number of avian species. In this paper, we will examine how different strategies for using a tool emerge during the simulated evolution of a two degree-of-freedom articulated limb in a reaching task environment. The limb is controlled by recurrent neural networks that are evolved using the NeuroEvolution of Augmenting Topologies (NEAT) algorithm, which allows for evolution of the topology of the network. First, we evolve controllers using two different fitness functions. One of these involves only very broad fitness criteria, such as the distance to target, while the other relies on more task knowledge, such as the difference between the number of required and actual tool pickups. Second, we observe that these fitness functions favor evolution of two distinct tool use strategies, and that the more informed fitness function leads to superior performance. Third, we compare the topological structure of the evolved neural circuits in detail, and relate behavioral differences to significant topological differences. Our results allow us to determine when the correct network topology for a given behavior has been found during evolution using NEAT, after which further changes to topology do not substantially improve fitness.

The fly visual system, although tiny when compared to the mammalian visual system, can still perform highly sophisticated spatial tasks like collision avoidance, landing on objects, pursuit of prey, etc. Flies outperform human-made autonomous flying systems in solving such spatial tasks by a long way. This is partly due to their ability to perceive and respond to optical flow generated by motion in the environment. They are also known to take actions that alter the image flow on their eyes. Higher level neurons in the fly visual system respond to different types of complex optical flows due to rotation and translation, by pooling information from local motion detectors called the elementary motion detectors (EMDs) in the lower level. In this sense, neuronal responses (spikes) from these optical flow detectors in the fly carry highly encoded signals: a single spike can represent a complex dynamical pattern of movement in the visual field. In this paper, we investigate how such highly encoded signals can be interpreted and utilized in the fly's brain, while solely operating on the encoded internal spike patterns within their brain and no direct external (unencoded) sensory information, i.e. a form of grounding. With a computational model of the optical flow detectors based on those in the fly, we show that specific pattern of action (or coordinated motor output) is the only way that a fly can decode its internal spikes and generate meaningful, relevant behavior based on that.

Recurrent neural networks (RNNs) often show very complicated temporal behavior. In this paper, we investigate the dynamics of a simple recurrent neural network used in a nontrivial articulated limb control task in a tool use domain. The RNN for the task is evolved by the NeuroEvolution of Augmenting Topologies (NEAT) algorithm. As a non-linear dynamical system, RNNs exhibit strong correlation between their external behavior and internal dynamics. Discovery of the fixed points and limit cycles in the system dynamics will help us understand the roles of neurons and connections in the neural network, and provide insights on how to analyze the function of biology-based nervous systems. We hope our results can provide new criteria to the evaluation of the evolved RNNs, and help the diagnosis of the failures in control problems using neuroevolution.

“Happy New Year!” At the beginning of 2016, I would like to take this opportunity to wish everyone a very happy, healthy, and prosperous new year! It is my great honor and privilege to serve as the Editor-in-Chief (EiC) of the IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (TNNLS), and I am excited to write this Editorial to start a new journey with you all.

Rapid advances in high-resolution, high-throughput 3D microscopy techniques in the past decade have opened up new avenues for brain research. One such technique developed in our lab is called the Knife-Edge Scanning Microscopy (KESM). The basic principle of KESM is to line-scan image while simultaneously sectioning thin tissue blocks using a diamond microtome. We have successfully sectioned and imaged whole mouse brains and portions of a rat brain processed with different stains to investigate the microstructures within. In this paper, we will present a fully automated soma (cell body) detection method based on random forests, working on Nissl-stained rat brain specimen. The method enables fast and accurate cell counting and density measurement in different brain regions.

Tool use requires high levels of cognitive function and is only observed in higher mammals and some avian species such as corvids. In this paper, we will investigate how the capability to use tools can spontaneously emerge in a simulated evolution of a two degree-of-freedom articulated limb. The controller for the limb was evolved as neural circuits that can gradually take on arbitrary topologies (NeuroEvolution of Augmenting Topologies, or NEAT). First, we show how very broad fitness criteria such as distance to the target, number of steps to reach the target, and tool pick-up frequency are enough to give rise to tool using behavior. Second, we analyze the evolved neural circuits to find properties that enable tool use. We expect our results to help us understand the origin of tool use and the kind of neural circuits that enabled such a powerful trait.

This paper presents a bioinspired digital liquid-state machine (LSM) for low-power very-large-scale-integration (VLSI)-based machine learning applications. To the best of the authors' knowledge, this is the first work that employs a bioinspired spike-based learning algorithm for the LSM. With the proposed online learning, the LSM extracts information from input patterns on the fly without needing intermediate data storage as required in offline learning methods such as ridge regression. The proposed learning rule is local such that each synaptic weight update is based only upon the firing activities of the corresponding presynaptic and postsynaptic neurons without incurring global communications across the neural network. Compared with the backpropagation-based learning, the locality of computation in the proposed approach lends itself to efficient parallel VLSI implementation. We use subsets of the TI46 speech corpus to benchmark the bioinspired digital LSM. To reduce the complexity of the spiking neural network model without performance degradation for speech recognition, we study the impacts of synaptic models on the fading memory of the reservoir and hence the network performance. Moreover, we examine the tradeoffs between synaptic weight resolution, reservoir size, and recognition performance and present techniques to further reduce the overhead of hardware implementation. Our simulation results show that in terms of isolated word recognition evaluated using the TI46 speech corpus, the proposed digital LSM rivals the state-of-the-art hidden Markov-model-based recognizer Sphinx-4 and outperforms all other reported recognizers including the ones that are based upon the LSM or neural networks.

Accurate estimation of neuronal count and distribution is central to the understanding of the organization and layout of cortical maps in the brain, and changes in the cell population induced by brain disorders. High-throughput 3D microscopy techniques such as Knife-Edge Scanning Microscopy (KESM) are enabling whole-brain survey of neuronal distributions. Data from such techniques pose serious challenges to quantitative analysis due to the massive, growing, and sparsely labeled nature of the data. In this paper, we present a scalable, incremental learning algorithm for cell body detection that can address these issues. Our algorithm is computationally efficient (linear mapping, non-iterative) and does not require retraining (unlike gradient-based approaches) or retention of old raw data (unlike instance-based learning). We tested our algorithm on our rat brain Nissl data set, showing superior performance compared to an artificial neural network-based benchmark, and also demonstrated robust performance in a scenario where the data set is rapidly growing in size. Our algorithm is also highly parallelizable due to its incremental nature, and we demonstrated this empirically using a MapReduce-based implementation of the algorithm. We expect our scalable, incremental learning approach to be widely applicable to medical imaging domains where there is a constant flux of new data.

In pattern recognition, data integration is an important issue, and when properly done, it can lead to improved performance. Also, data integration can be used to help model and understand multimodal processing in the brain. Amari proposed α-integration as a principled way of blending multiple positive measures (e.g., stochastic models in the form of probability distributions), enabling an optimal integration in the sense of minimizing the α-divergence. It also encompasses existing integration methods as its special case, for example, a weighted average and an exponential mixture. The parameter α determines integration characteristics, and the weight vector w assigns the degree of importance to each measure. In most work, however, α and w are given in advance rather than learned. In this letter, we present a parameter learning algorithm for learning α and ω from data when multiple integrated target values are available. Numerical experiments on synthetic as well as real-world data demonstrate the effectiveness of the proposed method.

Recent Reinforcement Learning (RL) algorithms, such as R-MAX, make (with high probability) only a small number of poor decisions. In practice, these algorithms do not scale well as the number of states grows because the algorithms spend too much effort exploring. We introduce an RL algorithm State TArgeted R-MAX (STAR-MAX) that explores a subset of the state space, called the exploration envelope ξ. When ξ equals the total state space, STAR-MAX behaves identically to R-MAX. When ξ is a subset of the state space, to keep exploration within ξ, a recovery rule β is needed. We compared existing algorithms with our algorithm employing various exploration envelopes. With an appropriate choice of ξ, STAR-MAX scales far better than existing RL algorithms as the number of states increases. A possible drawback of our algorithm is its dependence on a good choice of ξ and β. However, we show that an effective recovery rule β can be learned on-line and ξ can be learned from demonstrations. We also find that even randomly sampled exploration envelopes can improve cumulative rewards compared to R-MAX. We expect these results to lead to more efficient methods for RL in large-scale problems.

In this paper, we will review a novel microscopy modality called Knife-Edge Scanning Microscopy (KESM) that we have developed over the past twelve years (since 1999) and discuss its relevance to connectomics and neural networks research. The operational principle of KESM is to simultaneously section and image small animal brains embedded in hard polymer resin so that a near-isotropic, sub-micrometer voxel size of 0.6 μm × 0.7 μm × 1.0 μm can be achieved over ~1 cm3volume of tissue which is enough to hold an entire mouse brain. At this resolution, morphological details such as dendrites, dendritic spines, and axons are visible (for sparse stains like Golgi). KESM has been successfully used to scan whole mouse brains stained in Golgi (neuronal morphology), Nissl (somata), and India ink (vasculature), providing unprecedented insights into the system-level architectural layout of microstructures within the mouse brain. In this paper, we will present whole-brain-scale data sets from KESM and discuss challenges and opportunities posed to connectomics and neural networks research by such detailed yet system-level data.

Sensory data integration is an important task in human brain for multimodal processing as well as in machine learning for multisensor processing. α-integration was proposed by Amari as a principled way of blending multiple positive measures (e.g., stochastic models in the form of probability distributions), providing an optimal integration in the sense of minimizing the α-divergence. It also encompasses existing integration methods as its special case, e.g., weighted average and exponential mixture. In α-integration, the value of α determines the characteristics of the integration and the weight vector w assigns the degree of importance to each measure. In most of the existing work, however, α and w are given in advance rather than learned. In this paper we present two algorithms, for learning α and w from data when only a few integrated target values are available. Numerical experiments on synthetic as well as real-world data confirm the proposed method's effectiveness.

In pattern recognition, data integration is a processing method to combine multiple sources so that the combined result can be more accurate than a single source. Evidence theory is one of the methods that have been successfully applied to the data integration task. Since Dempster-Shafer theory as the first evidence theory can be against our intuitive reasoning with some data sets, many researchers have proposed different rules for evidence theory. Among all these rules, the averaging rule is known to be better than others. On the other hand, a-integration was proposed by Amari as a principled way of blending multiple positive measures. It is a generalized averaging algorithm including arithmetic, geometric and harmonic means as its special case. In this paper, we generalize evidence theory with α-integration. Our experimental results show how our proposed methods work.

Tactile receptive fields (RFs) are similar to visual receptive fields, while there is a subtle difference. Our previous work showed that tactile RFs have advantage in texture boundary detection tasks compared to visual RFs. Our working hypothesis was that tactile RFs are better in texture tasks since texture is basically a surface property, more intimately linked with touch than with vision. From an information processing point of view, touch and vision are very similar (i.e., two-dimensional sensory surface). Then, the question is what drives the two types of RFs to become different? In this paper, we investigated the possibility that tactile RF and visual RF emerge based on an identical cortical learning process, where the only difference is in the input type, natural-scene-like vs. texture-like. We trained a self-organizing map model of the cortex (the LISSOM model) on two different kinds of input, (1) natural scene and (2) texture, and compared the resulting RFs. The main result is that RFs trained on natural scenes have RFs resembling visual RFs, while those trained on texture resemble tactile RFs. These results suggest that the type of input most commonly stimulating the sensory modality (natural scene for vision and texture for touch), and not the intrinsic organization of the sensors or the developmental process in the cortex, determine the RF property. We expect these results to shed new light on the differences and similarities between touch and vision.

3D reconstruction on dense nanoscale medical images is a very challenging research topic. The challenge comes from the fact that boundaries of objects on such images are not always very clear due to imperfect staining. This makes the segmentation of dense nanoscale medical images very difficult and thus increases the difficulty in 3D reconstruction. In this paper, we proposed a method based on watershed and an interactive segmentation technique, graph cuts, to extract 3D volumes from dense nanoscale medical images. In our method, images are first segmented by a marker-controlled watershed algorithm. Markers for watershed segmentation algorithm are seed points generated by using distance transform, followed by a new grouping method that clusters seed points that are too close. Regions obtained by watershed transform segmentation algorithms are considered as nodes in a graph. Edges are to connect between the nodes in adjacent image slices. The weight on each edge is defined based on the overlapped area between nodes. User-selected nodes (regions) in an initial image slice serve as hard constraints in the minimization process. A globally optimal 3D volume is obtained by minimizing MAP-MRF energy function via graph cuts. In our application, in order to obtain a complete 3D volume structures including branching, the final 3D volume is the union of two 3D volumes obtained by performing the minimization of MAP-MRF energy function using graph cuts forwards and backwards through the image stack. Experiments are conducted both on synthetic data and on nanoscale image sequences from the Serial Block Face Scanning Electron Microscope (SBF-SEM). The results show that our method can successfully extract 3D volumes.

Early primitive animals with simple feed-forward neuronal circuits were limited to reactive behavior. Through evolution, they were gradually equipped with memory and became able to utilize information from the past. Such memory is usually implemented with recurrent connections and certain behavioral changes are thought to precede the reconstitution of the neuronal circuit's topology. If so, what could have been the behavior to drive such a rewiring? Our hypothesis is that the secretion and detection of chemical markers in the environment could be a precursor of internal memory. We will show how memory-like behavior can be expressed in memoryless reactive agents by taking advantage of the external chemical markers. Our results show that given chemical marker use, reactive agents are able to develop intelligent strategies in solving a biologically plausible food foraging task requiring spatial memory. We also found interesting analogy between the evaporation of the chemical markers and the recency effect in memory and how it affects the foraging strategy. These results are expected to help us better understand the possible evolutionary route from reactive to cognitive agents.

A large number of neural network models are based on a feedforward topology (perceptrons, backpropagation networks, radial basis functions, support vector machines, etc.), thus lacking dynamics. In such networks, the order of input presentation is meaningless (i.e., it does not affect the behavior) since the behavior is largely reactive. That is, such neural networks can only operate in the present, having no access to the past or the future. However, biological neural networks are mostly constructed with a recurrent topology, and recurrent (artificial) neural network models are able to exhibit rich temporal dynamics, thus time becomes an essential factor in their operation. In this paper, we will investigate the emergence of recollection and prediction in evolving neural networks. First, we will show how reactive, feedforward networks can evolve a memory-like function (recollection) through utilizing external markers dropped and detected in the environment. Second, we will investigate how recurrent networks with more predictable internal state trajectory can emerge as an eventual winner in evolutionary struggle when competing networks with less predictable trajectory show the same level of behavioral performance. We expect our results to help us better understand the evolutionary origin of recollection and prediction in neuronal networks, and better appreciate the role of time in brain function.

The blood vessels in the retina have a characteristic radiating pattern, while there exists a significant variation dependent on the individual and/or medical condition. Extracting the geometric properties of these blood vessels have several important applications, such as biometrics (for identification) and medical diagnosis. In this paper, we will focus on biometric applications. For this, we propose a fast and accurate algorithm for tracing the blood vessels, and compare several candidate summary features based on the tracing results. Existing tracing algorithms based on a detailed analysis of the image can be too slow to quickly process a large volume of retinal images in real time (e.g., at a security check point). In order to select good features that can be extracted from the traces, we used kernel Isomap to test the distance between different retinal images as projected onto their respective feature spaces. We tested the following feature set: (1) angle among branches, (2) the number of fiber based on distance, (3) distance between branches, and (4) inner product among branches. Our results indicate that features 3 and 4 are prime candidates for use in fast, realtime biometric tasks. We expect our method to lead to fast and accurate biometric systems based on retinal images.

Texture segmentation is an effortless process in scene analysis, yet its mechanisms have not been sufficiently understood. A common assumption in most current approaches is that texture segmentation is a vision problem. However, considering that texture is basically a surface property, this assumption can at times be misleading. One interesting possibility is that texture may be more intimately related with touch than with vision. Recent neurophysiological findings showed that receptive fields for touch resemble that of vision, albeit with some subtle differences. To leverage on this, we tested how such distinct properties in tactile receptive fields can affect texture segmentation performance, as compared to that of visual receptive fields. Our main results suggest that touch has an advantage over vision in texture processing. We expect our findings to shed new light on the role of tactile perception of texture and its interaction with vision, and help develop more powerful, biologically inspired texture segmentation algorithms.

Speaker independent feature extraction is a critical problem in speech recognition. Oriented principal component analysis (OPCA) is a potential solution that can find a subspace robust against noise of the data set. The objective of this paper is to find a speaker-independent subspace by generalizing OPCA in two steps: First, we find a nonlinear subspace with the help of a kernel trick, which we refer to as kernel OPCA. Second, we generalize OPCA to problems with more than two phonemes, which leads to oriented discriminant analysis (ODA). In addition, we equip ODA with the kernel trick again, which we refer to as kernel ODA. The models are tested on the CMU ARCTIC speech database. Our results indicate that our proposed kernel methods can outperform linear OPCA and linear ODA at finding a speaker-independent phoneme space.

MOTIVATION: Reverse engineering of genetic regulatory networks from experimental data is the first step toward the modeling of genetic networks. Linear state-space models, also known as linear dynamical models, have been applied to model genetic networks from gene expression time series data, but existing works have not taken into account available structural information. Without structural constraints, estimated models may contradict biological knowledge and estimation methods may over-fit. RESULTS: In this report, we extended expectation-maximization (EM) algorithms to incorporate prior network structure and to estimate genetic regulatory networks that can track and predict gene expression profiles. We applied our method to synthetic data and to SOS data and showed that our method significantly outperforms the regular EM without structural constraints. AVAILABILITY: The Matlab code is available upon request and the SOS data can be downloaded from http://www.weizmann.ac.il/mcb/UriAlon/Papers/SOSData/, courtesy of Uri Alon. Zak's data is available from his website, http://www.che.udel.edu/systems/people/zak.

Neural conduction delay is a serious issue for organisms that need to act in real time. Various forms of flash-lag effect (FLE) suggest that the nervous system may perform extrapolation to compensate for delay. For example, in motion FLE, the position of a moving object is perceived to be ahead of a brief flash when they are actually colocalized. However, the precise mechanism for extrapolation at a single-neuron level has not been fully investigated. Our hypothesis is that facilitating synapses, with their dynamic sensitivity to the rate of change in the input, can serve as a neural basis for extrapolation. To test this hypothesis, we constructed and tested models of facilitating dynamics. First, we derived a spiking neuron model of facilitating dynamics at a single-neuron level, and tested it in the luminance FLE domain. Second, the spiking neuron model was extended to include multiple neurons and spike-timing-dependent plasticity (STDP), and was tested with orientation FLE. The results showed a strong relationship between delay compensation, FLE, and facilitating synapses/STDP. The results are expected to shed new light on real time and predictive processing in the brain, at the single neuron level.

Our earlier work has suggested that neuronal transmission delay may cause serious problems unless a compensation mechanism exists. In that work, facilitating neuronal dynamics was found to be effective in battling delay (the facilitating activation network model, or FAN). A systematic analysis showed that the previous FAN model has a subtle problem especially when high facilitation rates are used. We derived an improved facilitating dynamics at the neuronal level to overcome this limitation. In this paper, we tested our proposed approach in 2D pole balancing controllers, where it was shown to perform better than the previous FAN model. We also systematically tested the correlation between delay duration on the one hand and facilitation rate that effectively overcome the increasing delay on the other hand. Finally, we investigated the differential utilization of facilitating dynamics in sensory vs. motor neurons and found that motor neurons utilize the facilitating dynamics more than the sensory neurons. These findings are expected to help us better understand the role of facilitation in natural and artificial agents.

Delay in the nervous system is a serious issue for an organism that needs to act in real time. For example, during the time a signal travels from a peripheral sensor to the central nervous system, a moving object in the environment can cover a significant distance which can lead to critical errors in the effect of the corresponding motor output. This paper proposes that facilitating synapses which show a dynamic sensitivity to the changing input may play an important role in compensating for neural delays, through extrapolation. The idea was tested in a modified 2D pole-balancing problem which included sensory delays. Within this domain, we tested the behavior of recurrent neural networks with facilitatory neural dynamics trained via neuroevolution. Analysis of the performance and the evolved network parameters showed that, under various forms of delay, networks utilizing extrapolatory dynamics are at a significant competitive advantage compared to networks without such dynamics. In sum, facilitatory (or extrapolatory) dynamics can be used to compensate for delay at a single-neuron level, thus allowing a developing nervous system to stay in touch with the present environmental state.

In orientation flash-lag effect (FLE), a continuously rotating bar in the center is perceived to be misaligned toward the direction of rotation when compared to a briefly flashed pair of flanking bars that are actually aligned. The implication of this simple visual illusion is quite profound: The effect may be due to motion extrapolation, undoing the effects of neural conduction delay. Previously, we showed that facilitating synapses may be a neural basis of such a delay compensation mechanism in other forms of FLE such as luminance FLE. However, the approach based on a single neuron cannot be applied to orientation FLE since firing rate in a single neuron cannot represent the full range of orientations. Here, we extend our model to multiple neurons, and show that facilitating synapses, together with adaptation through Spike-Timing-Dependent Plasticity (STDP), can serve as a neural basis for delay compensation giving rise to orientation FLE.

A stationary display of white discs positioned on intersecting gray bars on a dark background gives rise to a striking scintillating effect—the scintillating grid illusion. The spatial and temporal properties of the illusion are well known, but a neuronal-level explanation of the mechanism has not been fully investigated. Motivated by the neurophysiology of the Limulus retina, we propose disinhibition and self-inhibition as possible neural mechanisms that may give rise to the illusion. In this letter, a spatiotemporal model of the early visual pathway is derived that explicitly accounts for these two mechanisms. The model successfully predicted the change of strength in the illusion under various stimulus conditions, indicating that low-level mechanisms may well explain the scintillating effect in the illusion.

In flash-lag effect (FLE), the position of a moving object is perceived to be ahead of a brief flash when they are actually co-localized. This phenomenon may be due to motion extrapolation: the nervous system has internal conduction delay, thus signals received with a delay in central areas have to be extrapolated for the internal state to be temporally aligned with that of the environment. The precise neural mechanism of such a process has not been fully investigated. Here, we propose that facilitating synapses can be a potential candidate. We tested this idea in FLE and showed that our model behaviour is consistent with experimental data. In sum, facilitatory neural dynamics may underlie delay compensation, thus giving rise to FLE.

How multiple specialized cortical areas in the brain interact with each other to give rise to an integrated behavior is a largely unanswered question. This paper proposes that such an integration can be understood under the framework of analogy and that part of the thalamus and the thalamic reticular nucleus (TRN) may be playing a key role in this respect. The proposed thalamocortical model of analogy heavily depends on a diverse set of temporal parameters including axonal delay and membrane time constant, each of which is critical for the proper functioning of the model. The model requires a specific set of conditions derived from the need of the model to process analogies. Computational results with a network of integrate and fire (IF) neurons suggest that these conditions are indeed necessary, and furthermore, data found in the experimental literature also support these conditions. The model suggests that there is a very good reason for each temporal parameter in the thalamocortical network having a particular value, and that to understand the integrated behavior of the brain, we need to study these parameters simultaneously, not separately.

The corticothalamic feedback and the thalamic reticular nucleus have gained much attention lately because of their integrative and modulatory functions. A previous study by the author suggested that this circuitry could process analogies (i.e., the analogy hypothesis). In this paper, the proposed model was implemented as a network of leaky integrate-and-fire neurons to test the analogy hypothesis. The previous proposal required specific delay and temporal dynamics, and the implemented network tuned accordingly functioned as predicted. Furthermore, these specific conditions turn out to be consistent with experimental data, suggesting that a further investigation of the thalamocortical circuit within the analogical framework may be worthwhile.

How does our visual system detect prominent contours? Our investigation begins with the observation that neurons in the visual cortex have receptive fields similar to oriented Gabor filters. Unlike plain gray-level intensity histograms which greatly vary across images, we found that Gabor orientation-response (or orientation-energy) histograms of natural images have a fairly uniform shape. Based on this observation, we derived a threshold criterion which only depends on the standard deviation of the orientation-energy distribution. Thus, the same principle could be uniformly applied to different natural images, either locally or globally. Comparison with thresholds chosen by humans showed that the criterion can accurately predict human performance. Further, the proposed criterion can be easily implemented in a neural network, which is currently under investigation.