Modern Spiking Neural Networks (SNNs) employ efficient neuron models to scale networks using Deep Learning techniques. However, as conventional artificial neural networks continue to outperform and outpace SNNs, research shifted towards efficiency, thereby overlooking the question of performance. This document explores the idea of finding a computational edge for SNNs in greater biophysical realism. The authors’ inspiration lies in the known dependency of biological spike activity on an interplay of continuous flows of ions. Current spiking neuron models adopt a simplified representation of this principle by either not considering ions (Integrate-and-Fire) or viewing their concentrations as static quantities (Hodgkin-Huxley). After presenting some of the complexity surrounding biological neurons, the authors investigate intricate molecular mechanisms as well as non-stationary flows, incorporated under the proposed Free-Flowing Ion Concentrations (FFIC) model framework. Their importance for neuronal function is studied from a computational lens. Supported by novel descriptive techniques tailored for spike activity, computational properties of FFIC models are examined first within individual neurons and then as part of cortical microcircuit-like SNNs. Results indicate that the combination of these unconventional ion dynamics consistently leads to affluent and diverse neuronal activity, able to express a range of rich and complex behaviours. When connected together, FFIC neurons with strongly coupled molecular mechanisms imbue SNNs with higher information processing capacity. The stationarity of concentrations is seen to influence their receptivity to different types of stimulation (current or spikes) and the expression of complex transfer functions.

Dialogue State Tracking (DST) is a core component towards task oriented dialogue system. It fills manually-set slots at each turn of an utterance, which indicate the current topics or user requirement. In this work we propose a memory based state tracker that includes a memory encoder which encodes the dialogue history into a memory vector, and then connects to a pointer network which makes predictions. Our model reached a joint goal accuracy of 49.16% on MultiWOZ 2.0 data set (Budzianowski et al., 2018) and 47.27% on MultiWOZ 2.1 data set (Eric et al., 2019), outperforming the benchmark result.

Abstract We explore the effect of connectivity and topology on the dynamical behaviour of Reservoir Computers. At present, considerable effort is taken to design and hand-craft physical reservoir computers. Both structure and physical complexity are often pivotal to task performance, however, assessing their overall importance is challenging. Using a recently developed framework, we evaluate and compare the dynamical freedom (referring to quality) of neural network structures, as an analogy for physical systems. The results quantify how structure affects the behavioural range of networks. It demonstrates how high quality reached by more complex structures is often also achievable in simpler structures with greater network size. Alternatively, quality is often improved in smaller networks by adding greater connection complexity. This work demonstrates the benefits of using dynamical behaviour to assess the quality of computing substrates, rather than evaluation through benchmark tasks that often provide a narrow and biased insight into the computing quality of physical systems.

In this work, we trained different bilingual word embeddings models without word alignments (BilBOWA) using linear Bag-of-words contexts and dependency-based contexts. BilBOWA embedding models learn distributed representations of words by jointly optimizing a monolingual and a bilingual objective. Including dependency features in the monolingual objective, improves the accuracy of learning bilingual word embeddings up to 6% points in English-Spanish (En-Es) and up to 2.5% points in English-German (En-De) language pairs in word translation task compared to the baseline model. However, using these dependency features in both monolingual and bilingual objectives does not lead to any improvement in the En-Es language pair and only shows minor improvement for En-De. Moreover, our results provide evidence that using dependency features in bilingual word embeddings has a different effect based on syntactic and sentence structure similarity of the language pair.

In this paper, we introduce a theoretical basis for a Hadoop-based neural network for parallel and distributed feature selection in Big Data sets. It is underpinned by an associative memory (binary) neural network which is highly amenable to parallel and distributed processing and fits with the Hadoop paradigm. There are many feature selectors described in the literature which all have various strengths and weaknesses. We present the implementation details of five feature selection algorithms constructed using our artificial neural network framework embedded in Hadoop YARN. Hadoop allows parallel and distributed processing. Each feature selector can be divided into subtasks and the subtasks can then be processed in parallel. Multiple feature selectors can also be processed simultaneously (in parallel) allowing multiple feature selectors to be compared. We identify commonalities among the five features selectors. All can be processed in the framework using a single representation and the overall processing can also be greatly reduced by only processing the common aspects of the feature selectors once and propagating these aspects across all five feature selectors as necessary. This allows the best feature selector and the actual features to select to be identified for large and high dimensional data sets through exploiting the efficiency and flexibility of embedding the binary associative-memory neural network in Hadoop.

MOTIVATION: The identification of suitable conditions for crystallization is a rate-limiting step in protein structure determination. The pH of an experiment is an important parameter and has the potential to be used in data-mining studies to help reduce the number of crystallization trials required. However, the pH is usually recorded as that of the buffer solution, which can be highly inaccurate. RESULTS: Here, we show that a better estimate of the true pH can be predicted by considering not only the buffer pH but also any other chemicals in the crystallization solution. We use these more accurate pH values to investigate the disputed relationship between the pI of a protein and the pH at which it crystallizes. AVAILABILITY AND IMPLEMENTATION: Data used to generate models are available as Supplementary Material. CONTACT: [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

In recent years, the range of sensing technologies has expanded rapidly, whereas sensor devices have become cheaper. This has led to a rapid expansion in condition monitoring of systems, structures, vehicles, and machinery using sensors. Key factors are the recent advances in networking technologies such as wireless communication and mobile ad hoc networking coupled with the technology to integrate devices. Wireless sensor networks (WSNs) can be used for monitoring the railway infrastructure such as bridges, rail tracks, track beds, and track equipment along with vehicle health monitoring such as chassis, bogies, wheels, and wagons. Condition monitoring reduces human inspection requirements through automated monitoring, reduces maintenance through detecting faults before they escalate, and improves safety and reliability. This is vital for the development, upgrading, and expansion of railway networks. This paper surveys these wireless sensors network technology for monitoring in the railway industry for analyzing systems, structures, vehicles, and machinery. This paper focuses on practical engineering solutions, principally, which sensor devices are used and what they are used for; and the identification of sensor configurations and network topologies. It identifies their respective motivations and distinguishes their advantages and disadvantages in a comparative review.

Collective identity helps swarms remain coherent in the presence of others. Building identity into artificial systems enables groups of agents to work in the same area as one another, without interference from other agents. By linking the firefly algorithm to the control logic of the agents, we present a method to form and maintain identity in swarms. By measuring swarm polarization, and swarm overlap, we show that the inclusion of an identity allows a swarm to remain coherent for an extended period of time, without interference from other swarms.

Reliable feature detection is a prerequisite to higher level decisions regarding image content. In the domain of spectrogram track detection and classification, the detection problem is compounded by low signal-to-noise ratios and high variation in track appearance. Evaluation of standard feature detection methods in the literature is essential to determine their strengths and weaknesses in this domain. With this knowledge, improved detection strategies can be developed. This paper presents a comparison of line detectors and a novel, multi-scale, linear feature detector able to detect tracks of varying gradients. We outline improvements to the multi-scale search strategies which reduce run-time costs. It is shown that the Equal Error Rates of existing methods are high, highlighting the need for research into novel detectors. Results demonstrate that the proposed method offers an improvement in detection rates when compared to other, state of the art, methods whilst keeping false positive rates low. It is also shown that a multi-scale implementation offers an improvement over fixed scale implementations.

In this paper, a methodology for facial feature identification and localization approach is proposed based on binary neural network algorithms. We present a head pose and facial expression invariant 3D shape descriptor called Mesh-like Multi Circle Curvature Descriptor (MMCCD), which provides more 3D curvature attributes than other similar approaches. To search and match the feature patterns with more attributes, we use Advanced Uncertain Reasoning Architecture (AURA) k-Nearest Neighbour algorithms to encode, train and match the feature patterns based on 3D shape curvature. Experiments performed on the FRGC dataset (4950 3D faces) with pose and expression variations show that our approach is able to achieve an accurate (over 99.69% nose tip identification) and robust identification and localization of facial features.

This paper proposes an extension to the active contour algorithm for the detection of linear patterns within remote sensing and vibration data. The proposed technique uses an alternative energy force, overcoming the limitations of the original algorithm, which relies upon simple energy formulations to extract intensity and gradient information from an image. We overcome these by forming a noise model, which is used to detect a feature¿s presence, and by integrating information from several locations within an image to strengthen the detection process.

Association rule mining is a thoroughly studied problem in data mining. Its solution has been aimed for by approaches based on different strategies involving, for instance, the use of novel data structures to represent the knowledge discovered, the transformation of the input data to speed up the process, the exploitation of the itemset properties either to traverse the possible itemset search space optimally or to form compact representation of the frequent itemsets employed for the generation of the corresponding final rules, and others. Surprisingly, biologically-inspired approaches have rarely been proposed. In this work, we focus on investigating if a type of mapping neural network, better known as an associative memory, is suitable for association rule mining. In particular, our aim is to determine if itemset support can be estimated from the knowledge embedded in the weight matrix of a trained associative memory in order to generate further association rules from such a knowledge.

Twenty years ago a paper detailing a novel neural network, called ADAM and that could be directly implemented in hardware RAM, was published in the first conference of this series. Subsequent research based directly/indirectly on this type of RAM-based neural network founded a research group that has produced over 200 research documents. This paper overviews that research and goes on to mathematically define a CMML, a generalised version of a CMM (the component at the heart of ADAM). The CMML can be trained to replicate the exact computational properties of a CMM and so is a plug-and-play replacement to a CMM; whilst a different training algorithm gives it different properties when used in recall.

Frequent Itemset Mining, which is the core of Association Rule Mining, is a very well-known problem in the data mining field. Similarly, a Self-Organising Map is a well known neural network which has been used for data clustering mainly. In the discovery of frequent itemsets, conforming the raw material to create association rules, the support, being an itemset metric, is highly important since it determines the interestingness of any itemset in a mining process. In this work, we propose and define a probabilistic method to identify and extract from the weight matrix of a trained map the support of all of the possible itemsets that can be formed by the components of the patterns in the training dataset.

The AURA k-Nearest Neighbour classifier associates binary input and output vectors, forming a compact binary Correlation Matrix Memory (CMM). For a new input vector, matching vectors are retrieved and classification is performed on the basis of these recalled vectors. Real-world data is not binary and must therefore be encoded to form the required binary input. Efficient operation of the CMM requires that these binary input vectors are sparse. Current encoding of high dimensional data requires large vectors in order to remain sparse, reducing efficiency. This paper explores an alternative approach that produces shorter sparse codes, allowing more efficient storage of information without degrading the recall performance of the system.

Self-organizing maps are popular tools for data visualization and clustering. At the same time, due to the incorporation of new transactions, real-life databases change periodically. As a consequence of changes in our databases; our maps, which are derived from them, often become outdated and are therefore no longer usable for decision support. To tackle this problem, the application of incremental training methods has been suggested. The current incremental methods have been developed based on non-batch procedures. In this work, a batch-incremental-training algorithm for a self-organizing map is proposed. The results obtained are promising enough to affirm that the batch method might be considered for non-stationary environments.

This paper investigates the practical application of a new method called selfsimilar convolution masks to binary trademark images to produce affine invariant one dimensional histogram descriptions. Because the convolution mask is a scaled version of the original image, the normalised distribution histogram of the resulting grey scale image is an affine invariant description based purely upon the image structure, which can be then be used for database search using a database of binary images. The effectiveness of this approach to trademark image identification is tested. Multiple methods of representing the distribution histograms within the database are also tested for robustness to noise, generalisation and reliability. 1

An essential part of image analysis is the identification of objects within the image. This paper describes the results of applying a binary associative memory neural network (ADAM), to the complex task of identifying shapes in document images. The associative memory is used to implement the generalised Hough Transform, exploiting the fast look-up ability of the associative memory to give a high-speed image analysis tool.