Graph Neural Networks (GNNs) are the most common approach for learning complex relational data represented using graph data structures. Although GNNs are effective at learning representations of both nodes and graphs for a given task, the learning process is computationally expensive and as such, time and energy-inefficient. This paper investigates this challenge within the context of recent work on untrained graph representations that only train the solver model. We present Graph Vector Function Architecture (GVFA), a novel alternative to learning graph representations in GNNs that is based on hyperdimensional computing (HDC) principles. GVFA is a general zero-shot approach for graph and node representations without learning. As such, our representations are not task-specific and the computational costs of constructing them is substantially lower compared to learning-based GNN. Empirically, we demonstrate the expressiveness and generalization properties of different GVFA configurations. Our experimental results demonstrate that GVFA outperforms several classic GNNs on their benchmark datasets in terms of classification accuracy for both graph and node classification tasks, while also yielding a substantial reduction in training time.

Large Language Models (LLMs) are leading the Generative Artificial Intelligence transformation in natural language understanding. Beyond language understanding, LLMs have demonstrated capabilities in reasoning tasks, including commonsense, logical, and mathematical reasoning. However, their proficiency in causal understanding has been limited due to the complex nature of causal reasoning. Several recent studies have discussed the role of external causal models for improved causal understanding. Building on the success of Retrieval-Augmented Generation (RAG) for factual reasoning in LLMs, this paper introduces a novel approach that utilizes Causal Graphs as external sources for establishing causal relationships between complex vectors. This method is empirically evaluated using two benchmark datasets across the metrics of Context Relevance, Answer Relevance, and Grounding, in its ability to retrieve relevant context with causal alignment. The retrieval effectiveness is further compared with traditional RAG methods that are based on semantic proximity.

The vector superposition operation plays a central role in Hyperdimensional Computing (HDC), enabling compositionality of hypervectors without expanding the dimensionality, unlike concatenation. However, a problem arises when the quantity of superimposed vectors surpasses a certain threshold, which is determined by the hypervector’s information capacity relative to its dimensionality. Beyond this point, cross-talk noise incrementally obscures the distinctiveness of individual hypervectors and information is lost. To solve this challenge, we introduce a novel method for weighting individual hypervectors within the superposition, ensuring that only those hypervectors crucial for a given task are prioritized. The weights are learned end-to-end using the backpropagation algorithm in a neural network. Our method is characterized by two key features: (1) The resultant weighting model is exceptionally compact, as the number of trainable weights is equal to the total number of hypervectors in the superposition; (2) The model offers enhanced explainability due to the compositional nature of its encoding. These features collectively contribute to the efficiency and effectiveness of our proposed classification approach using hyperdimensional computing. We illustrate our approach through the multi-channel time series classification task. In this framework, each channel is encoded as a hypervector-descriptor, and those are subsequently composed into a single hypervector via superposition. This superimposed vector forms the basis for training the classification model based on the neural network. Applying our approach of weighted superposition on this task improved the classification performance compared to standard superposition or concatenation of feature vectors, especially for larger numbers of channels.

Motivated by recent innovations in biologically inspired neuromorphic hardware, this article presents a novel unsupervised machine learning algorithm named Hyperseed that draws on the principles of vector symbolic architectures (VSAs) for fast learning of a topology preserving feature map of unlabeled data. It relies on two major operations of VSA, binding and bundling. The algorithmic part of Hyperseed is expressed within the Fourier holographic reduced representations (FHRR) model, which is specifically suited for implementation on spiking neuromorphic hardware. The two primary contributions of the Hyperseed algorithm are few-shot learning and a learning rule based on single vector operation. These properties are empirically evaluated on synthetic datasets and on illustrative benchmark use cases, IRIS classification, and a language identification task using the n -gram statistics. The results of these experiments confirm the capabilities of Hyperseed and its applications in neuromorphic hardware.

The increasing use of Vector Symbolic Architectures (VSA) in machine learning has contributed towards en-ergy efficient computation, short training cycles and improved performance. A further advancement of VSA is to leverage sparse representations, where the VSA-encoded hypervectors are sparsified to represent receptive field properties when encoding sensory inputs. The hyperseed algorithm is an unsupervised machine learning algorithm based on VSA for fast learning a topology preserving feature map of unlabelled data. In this paper, we implement two methods of sparse block-codes on the hyperseed algorithm, they are selecting the maximum element of each block and selecting a random element of each block as the nonzero element. Finally, the sparsified hyperseed algorithm is empirically evaluated for performance using three distinct bench-mark datasets, Iris classification, classification and visualisation of synthetic datasets from the Fundamental Clustering Problems Suite and language classification using n-gram statistics.

Sequence-based methods for visual place recognition (VPR) have great importance due to their ability of additional information capture through the sequences compared to single image comparison. Vector symbolic architecture (VSA) started to gain attention within these methods due to the unique capabilities for representing variable-length sequences using single high-dimensional vectors. But the effect of different sequence parameters for the visual place recognition task is yet to be explored. In this work, we explore the parametrization of sequence encoding with VSA in the SeqNet variant of sequence-based visual place recognition and introduce a new hierarchical VPR method, which utilizes the proposed parametrization. We show that with our parametrization the VSA realization of sequence-based visual place recognition achieves on par results to conventional algorithms, while featuring the capability of being implemented on novel neuromorphic hardware for efficient execution.

The recent interest in federated learning has initiated the investigation for efficient models deployable in scenarios with strict communication and computational constraints. Furthermore, the inherent privacy concerns in decentralized and federated learning call for efficient distribution of information in a network of interconnected agents. Therefore, we propose a novel distributed classification solution that is based on shallow randomized networks equipped with a compression mechanism that is used for sharing the local model in the federated context. We make extensive use of hyperdimensional computing both in the local network model and in the compressed communication protocol, which is enabled by the binding and the superposition operations. Accuracy, precision, and stability of our proposed approach are demonstrated on a collection of datasets with several network topologies and for different data partitioning schemes.

(also known as Hyperdimensional Computing). This framework is well suited for implementation in stochastic, emerging hardware and it naturally expresses the types of cognitive operations required for Artificial Intelligence (AI). We demonstrate in this article that the field-like algebraic structure of Vector Symbolic Architectures offers simple but powerful operations on high-dimensional vectors that can support all data structures and manipulations relevant to modern computing. In addition, we illustrate the distinguishing feature of Vector Symbolic Architectures, "computing in superposition," which sets it apart from conventional computing. It also opens the door to efficient solutions to the difficult combinatorial search problems inherent in AI applications. We sketch ways of demonstrating that Vector Symbolic Architectures are computationally universal. We see them acting as a framework for computing with distributed representations that can play a role of an abstraction layer for emerging computing hardware. This article serves as a reference for computer architects by illustrating the philosophy behind Vector Symbolic Architectures, techniques of distributed computing with them, and their relevance to emerging computing hardware, such as neuromorphic computing.

We propose an approximation of echo state networks (ESNs) that can be efficiently implemented on digital hardware based on the mathematics of hyperdimensional computing. The reservoir of the proposed integer ESN (intESN) is a vector containing only n -bits integers (where is normally sufficient for a satisfactory performance). The recurrent matrix multiplication is replaced with an efficient cyclic shift operation. The proposed intESN approach is verified with typical tasks in reservoir computing: memorizing of a sequence of inputs, classifying time series, and learning dynamic processes. Such architecture results in dramatic improvements in memory footprint and computational efficiency, with minimal performance loss. The experiments on a field-programmable gate array confirm that the proposed intESN approach is much more energy efficient than the conventional ESN.

The emergence of low latency and high bandwidth 5G networks, alongside localized computation and data storage of edge computing are enabling real-time applications in industrial settings, such as smart grid, smart cities, and smart factories. The resolution, frequency and variety of data streams generated by such applications are not effectively processed and analysed by contemporary machine learning algorithms. This challenge is further complicated by the unlabelled and non-deterministic nature of the data streams. Hardware accelerated machine learning has been proposed to address some of these challenges but limited work has been published on unsupervised learning from unlabelled data. In this paper, we extend the hardware accelerated Self Organizing Map (SOM) algorithm by optimizing the learning rule for computational efficiency, followed by a comparative empirical evaluation with two other variants, tri-state SOM and integer SOM. We have used two datasets representative of real-time industrial applications in 5G networks and smart grids, for this evaluation.

Recent advances in Deep Learning have led to a significant performance increase on several NLP tasks, however, the models become more and more computationally demanding. Therefore, this paper tackles the domain of computationally efficient algorithms for NLP tasks. In particular, it investigates distributed representations of$n$-gram statistics of texts. The representations are formed using hyperdimensional computing enabled embedding. These representations then serve as features, which are used as input to standard classifiers. We investigate the applicability of the embedding on one large and three small standard datasets for classification tasks using nine classifiers. The embedding achieved on par$F_{1}$scores while decreasing the time and memory requirements by several times compared to the conventional$n$-gram statistics, e.g., for one of the classifiers on a small dataset, the memory reduction was 6.18 times; while train and test speed-ups were 4.62 and 3.84 times, respectively. For many classifiers on the large dataset, memory reduction was ca. 100 times and train and test speed-ups were over 100 times. Importantly, the usage of distributed representations formed via hyperdimensional computing allows dissecting strict dependency between the dimensionality of the representation and n-gram size, thus, opening a room for tradeoffs.

This paper investigates a combination of the two recently proposed techniques: superposition of multiple neural networks into one and neural network compression. We show that these two techniques can be successfully combined to deliver a great potential for trimming down deep convolutional neural networks. The work can be relevant in the context of implementing deep learning on low-end computing devices as it enables neural networks to fit edge devices with constrained computational resources (e.g. sensors, mobile devices, controllers). We study the trade-offs between the model compression rate and the accuracy of the superimposed tasks and present a CNN pipeline where the fully connected layers are isolated from the convolutional layers and serve as a general purpose neural processing unit for several CNN models. We show how deep models can be highly compressed with a limited accuracy degradation when additional compression is performed within the superposition principle.

The deployment of machine learning algorithms on resource-constrained edge devices is an important challenge from both theoretical and applied points of view. In this brief, we focus on resource-efficient randomly connected neural networks known as random vector functional link (RVFL) networks since their simple design and extremely fast training time make them very attractive for solving many applied classification tasks. We propose to represent input features via the density-based encoding known in the area of stochastic computing and use the operations of binding and bundling from the area of hyperdimensional computing for obtaining the activations of the hidden neurons. Using a collection of 121 real-world data sets from the UCI machine learning repository, we empirically show that the proposed approach demonstrates higher average accuracy than the conventional RVFL. We also demonstrate that it is possible to represent the readout matrix using only integers in a limited range with minimal loss in the accuracy. In this case, the proposed approach operates only on small n -bits integers, which results in a computationally efficient architecture. Finally, through hardware field-programmable gate array (FPGA) implementations, we show that such an approach consumes approximately 11 times less energy than that of the conventional RVFL.

Abstract A Bloom filter is a special case of an artificial neural network with two layers. Traditionally, it is seen as a simple data structure supporting membership queries on a set. The standard Bloom filter does not support the delete operation, and therefore, many applications use a counting Bloom filter to enable deletion. This paper proposes a generalization of the counting Bloom filter approach, called “autoscaling Bloom filters”, which allows adjustment of its capacity with probabilistic bounds on false positives and true positives. Thus, by relaxing the requirement on perfect true positive rate, the proposed autoscaling Bloom filter addresses the major difficulty of Bloom filters with respect to their scalability. In essence, the autoscaling Bloom filter is a binarized counting Bloom filter with an adjustable binarization threshold. We present the mathematical analysis of its performance and provide a procedure for minimizing its false positive rate.

The Self-Organizing Map algorithm has been proven and demonstrated to be a useful paradigm for unsupervised machine learning of two-dimensional projections of multidimensional data. The tri-state Self-Organizing Maps have been proposed as an accelerated resource-efficient alternative to the Self-Organizing Maps for implementation on field-programmable gate array (FPGA) hardware. This paper presents a generalization of the tri-state Self-Organizing Maps. The proposed generalization, which we call integer Self-Organizing Maps, requires only integer operations for weight updates. The presented experiments demonstrated that the integer Self-Organizing Maps achieve better accuracy in a classification task when compared to the original tri-state Self-Organizing Maps.

In cyber-physical petroleum systems (CPPS), accurate estimation of interwell connectivity is an important process to know reservoir properties comprehensively, determine water injection rate scientifically, and enhance oil recovery effectively for oil and gas (O&G) field. In this study, an artificial neural network (ANN) based analysis method is proposed to estimate interwell connectivity. The generated neural network is used to define the mapping function between production wells and surrounding injection wells based on the historical water injection and liquid production data. Finally, the proposed method is applied to a synthetic reservoir model. Experimental results show that ANN based approach is an efficient method for analyzing interwell connectivity.

The emergence of artificial intelligence (AI), empowered by robust computing infrastructure and abundance of data, maintains potential for radical transformation of human society, essentially a third phase in evolution. Numerous research endeavor, policy development, and thought-leadership are presently in progress aimed at discovering data-driven intelligent decision-making solutions for smart cities, smart grids, smart homes, and informed citizens as well as addressing potential risks posed by AI workplace automation. Joining this broad effort, this Special Section contributes six research articles that consolidate recent developments in AI for industrial informatics.

Hyperdimensional (HD) computing is a promising paradigm for future intelligent electronic appliances operating at low power. This paper discusses tradeoffs of selecting parameters of binary HD representations when applied to pattern recognition tasks. Particular design choices include density of representations and strategies for mapping data from the original representation. It is demonstrated that for the considered pattern recognition tasks (using synthetic and real-world data) both sparse and dense representations behave nearly identically. This paper also discusses implementation peculiarities which may favor one type of representations over the other. Finally, the capacity of representations of various densities is discussed.

The main contribution of this paper is a study of the applicability of hyperdimensional computing and learning with an associative memory for modeling the dynamics of complex automation systems. Specifically, the problem of learning an evidence-based model of a plant in a distributed automation and control system is considered. The model is learned in the form a finite state automata.

In this paper, we propose a new approach to implementing hierarchical graph neuron (HGN), an architecture for memorizing patterns of generic sensor stimuli, through the use of vector symbolic architectures. The adoption of a vector symbolic representation ensures a single-layer design while retaining the existing performance characteristics of HGN. This approach significantly improves the noise resistance of the HGN architecture, and enables a linear (with respect to the number of stored entries) time search for an arbitrary subpattern.

This article presents a methodology for intelligent, biologically inspired fault detection system for generic complex systems of systems. The proposed methodology utilizes the concepts of associative memory and vector symbolic architectures, commonly used for modeling cognitive abilities of human brain. Compared to classical methods of artificial intelligence used in the context of fault detection the proposed methodology shows an unprecedented performance, while featuring zero configuration and simple operations.

This paper presents a feasibility study of implementing parts of a distributed function block application as TinyOS modules running on Wireless Sensors as a part of Wireless Sensor Network. The paper first briefly describes underlying technologies and gives motivation for implementation of function blocks in TinyOS. The paper then presents implementation details about TinyOS realization of the one of the function block, which is a part of bigger distributed control application with the help of distributed function block application.

This article presents experiences of teachers from Luleâ University of Technology when enhancing the teaching approach and depth of an undergraduate course on network programming and distributed applications. During the trial run of the course in the fall of 2013 agent-oriented programming and cloud technologies were married to provide students an exciting practical scenario and capability to test the performance of truly large scale distributed systems under extremely high traffic loads.

The interoperability between various automation systems is considered as one of the major character of future automation systems. Service-oriented Architecture is a possible interoperability enabler between legacy and future automation systems. In order to prove the interoperability between those systems, a verification framework is essential. This paper proposes a configurable cloud-based validation environment for interoperability tests between various distributed automation systems. The testing framework is implemented in a multi-layer structure which provides automated closed-loop testing from the protocol level to the system level. The testing infrastructure is also capable for simulating automation systems as well as wireless sensor networks in the cloud. Test cases could be automatically generated and executed by the framework.

The concept of “Internet-of-Things” will undoubtedly emerge as the technology of the future. Educating specialists ready to bring the concept to the reality remains challenging in the scope of traditional university courses. The main challenge is how to enable students to think outside the boundaries of the particular discipline and therefore to enable the innovative thinking. This article describes an experiment with teaching Internet-of-Things as a common red thread across three courses which ran in parallel during fall semester 2012 at Luleå University of Technology in Sweden. We discuss the teaching methodology, the technology blocks which laid the ground for our teaching philosophy as well as the experiences and lessons learned.

Early 2012 Luleå University of Technology started a project on adopting cloud technology for implementation in the university's IT-infrastructure. This work-in-progress article describes the results of its pre-study phase aiming at understanding the feasibility of integrating and/or migrating main IT-infrastructure components into an IaaS system and opening ways for making university's resources more accessible to a wider public. Numbers of logistical, technical and education related challenges make such transition far from being trivial. The article focuses on the educational aspect of the pre-study. Specifically, work flows in education process of several courses in different disciplines in natural and engineering sciences were analyzed from the student and teacher perspectives. In the article a schematic of a sustainable IT infrastructure adjusted to the needs of higher education will be drafted. Further, technical readiness and challenges of using cloud technology for university scale IT-infrastructure are discussed.

In this article, we report on a mathematical model for the throughput and delay of LTE's Random Access Channel (RACH). The model is an improvement of a cell-level Markov model of Multichannel slotted ALOHA proposed previously in the literature. The improvements concern accounting for a possibility of contention resolution in the case when terminals select same preambles and distinguishing initially-transmitting nodes from retransmitting nodes. The improved model is verified and agrees with measurements obtained from a discrete-time event-based simulator of an LTE cell.

Distributed data storage is an important component of wireless sensor networks, which protects the mission critical information from unexpected node failures or malicious destruction of parts of the network. In this paper we present DISC, a protocol for distributed information storage and collection. The two major mechanisms in DISC which make our solution distinct from the related approaches are probabilistic choice of storing nodes and a search engine based on the usage of Bloom filters. In comparison to the deterministic choice of the backup node, the random selection strategy makes it virtually impossible for an attacker to determine and destroy the exact node keeping a particular piece of information. The usage of Bloom filters in the information search engine makes the navigation to a specific data fast and efficient. We show that with DISC the amount of recovered information is more than two times higher than that in deterministic storage schemes.