In this article, we present a novel approach that seamlessly integrates machine learning (ML) algorithms into wearable technology through the use of weightless neural networks (WNNs) and flexible integrated circuits (FlexICs). Our methodology employs combinational intelligent networks (COIN) for edge inference on resource-constrained devices, highlighting the advantages of WNNs in terms of power efficiency and minimal hardware requirements. We propose an automated design flow for implementing COIN as FlexICs aimed at developing scalable, cost-effective, and environmentally sustainable wearable monitoring solutions. As a proof-of-concept demonstrator, an arrhythmia detection FlexIC was fabricated using COIN to meet the stringent requirements of medium-complexity wearable applications, offering a promising path toward personalized and accessible healthcare solutions.

We introduce the Differentiable Weightless Neural Network (DWN), a model based on interconnected lookup tables. Training of DWNs is enabled by a novel Extended Finite Difference technique for approximate differentiation of binary values. We propose Learnable Mapping, Learnable Reduction, and Spectral Regularization to further improve the accuracy and efficiency of these models. We evaluate DWNs in three edge computing contexts: (1) an FPGA-based hardware accelerator, where they demonstrate superior latency, throughput, energy efficiency, and model area compared to state-of-the-art solutions, (2) a low-power microcontroller, where they achieve preferable accuracy to XGBoost while subject to stringent memory constraints, and (3) ultra-low-cost chips, where they consistently outperform small models in both accuracy and projected hardware area. DWNs also compare favorably against leading approaches for tabular datasets, with higher average rank. Overall, our work positions DWNs as a pioneering solution for edge-compatible high-throughput neural networks.

As Deep Neural Networks become more complex and computationally demanding, efficient models for inference at the edge, particularly multiplication-free ones, have gained significant attention. The Ultra Low-Energy Edge Neural Network (ULEEN) is a notable architecture optimized for high throughput edge designs. ULEEN uniquely employs Bloom Filters with binary values to compute neuron activation, boasting better efficiency metrics than Binary Neural Networks (BNNs). This work uncovers a gradient back-propagation bottleneck within ULEEN’s Bloom filters and introduces a simplified version of it as a solution: the "Soon Filter". Both theoretically and empirically, we demonstrate that our approach improves gradient back-propagation efficiency. Tests on MLPerf Tiny, MNIST and various UCI datasets reveal that our method surpasses ULEEN, BNN, and DeepShift. Notably, with MLPerf KWS (Key Word Spotting) dataset, we achieve 69.6% accuracy with only 101KiB, while ULEEN, BNN and DeepShift achieve only 67.4%, 55.9%, and 24.9% respectively. Remarkably, we also achieve 67.7% accuracy with only 50KiB, resulting in a 2x model size reduction compared to ULEEN while maintaining similar accuracy (+0.3%). This results underscores the promising potential of our solution for efficient inference at the edge in applications that rely on high throughput architectures.

We introduce Combinational Intelligent Networks (COIN), a machine learning technique that targets edge inference using low-resourced FPGAs or ASICs. COIN is an improvement on LogicWiSARD, a recent weightless neural network that achieves low power, small area, and high throughput. We convert the LogicWiSARD model into a binary neural network, train it using backpropagation, and then convert it to a COIN model. As a result, COIN can achieve higher accuracy than LogicWiSARD or it can require significantly fewer hardware resources when comparing models with similar accuracies. In comparison to a BNN implementation, FINN, small and large COIN models are more energy efficient demonstrating up to 11.5x higher inferences/Joule at similar accuracy. Our tool executes the complete flow, from training to RTL. and is publicly available.

Weightless neural networks are recognized for their online learning capacity and competitive performance with the state-of-the-art in different scenarios.Despite this, the literature has not adequately explored the potential of classification ensembles based on these models and their unique characteristics.This study introduces three types of ensembles based on the WiSARD weightless model and evaluates their effectiveness.The results show that these ensembles significantly improve accuracy compared to the WiSARD model and its ClusWiSARD extension, with a reasonable increase in computational cost.Furthermore, using ensembles eliminates the need for time-consuming tie-break policies of traditional WiSARD models.

Photovoltaic(PV) systems are one of the leading technologies to address climate change.Tracking systems improve energy generation by moving the surface to follow the sun's position however, these methods do not ensure optimal results in cloudy environments.This article proposes a closed-loop control algorithm for tracking based on reinforcement learning and weightless neural networks, compared to an astrological model.The method was applied in a single PV array on a single-axis tracking system, simulated with PVLib.Results showed that the architecture could improve results in cloudy environments but not in a clear-sky situation, as expected for a first approach.

Algorithms for mobile networking are increasingly being moved from centralized servers towards the edge in order to decrease latency and improve the user experience. While much of this work is traditionally done using ASICs, 6G emphasizes the adaptability of algorithms for specific user scenarios, which motivates broader adoption of FPGAs. In this paper, we propose the FPGA-based Weightless Intrusion Warden (FWIW), a novel solution for detecting anomalous network traffic on edge devices. While prior work in this domain is based on conventional deep neural networks (DNNs), FWIW incorporates a weightless neural network (WNN), a table lookup-based model which learns sophisticated nonlinear behaviors. This allows FWIW to achieve accuracy far superior to prior FPGA-based work at a very small fraction of the model footprint, enabling deployment on small, low-cost devices. FWIW achieves a prediction accuracy of 98.5% on the UNSW-NB15 dataset with a total model parameter size of just 192 bytes, reducing error by 7.9x and model size by 262x vs. LogicNets, the best prior edge-optimized implementation. Implemented on a Xilinx Virtex UltraScale+ FPGA, FWIW demonstrates a 59x reduction in LUT usage with a 1.6x increase in throughput. The accuracy of FWIW comes within 0.6% of the best-reported result in literature (Edge-Detect), a model several orders of magnitude larger. Our results make it clear that WNNs are worth exploring in the emerging domain of edge networking, and suggest that FPGAs are capable of providing the extreme throughput needed.

‘‘Extreme edge” 1 devices, such as smart sensors, are a uniquely challenging environment for the deployment of machine learning. The tiny energy budgets of these devices lie beyond what is feasible for conventional deep neural networks, particularly in high-throughput scenarios, requiring us to rethink how we approach edge inference. In this work, we propose ULEEN, a model and FPGA-based accelerator architecture based on weightless neural networks (WNNs). WNNs eliminate energy-intensive arithmetic operations, instead using table lookups to perform computation, which makes them theoretically well-suited for edge inference. However, WNNs have historically suffered from poor accuracy and excessive memory usage. ULEEN incorporates algorithmic improvements and a novel training strategy inspired by binary neural networks (BNNs) to make significant strides in addressing these issues. We compare ULEEN against BNNs in software and hardware using the four MLPerf Tiny datasets and MNIST. Our FPGA implementations of ULEEN accomplish classification at 4.0–14.3 million inferences per second, improving area-normalized throughput by an average of 3.6× and steady-state energy efficiency by an average of 7.1× compared to the FPGA-based Xilinx FINN BNN inference platform. While ULEEN is not a universally applicable machine learning model, we demonstrate that it can be an excellent choice for certain applications in energy- and latency-critical edge environments.

Weightless neural networks (WNNs) are a class of machine learning model which use table lookups to perform inference, rather than the multiply-accumulate operations typical of deep neural networks (DNNs). Individual weightless neurons are capable of learning non-linear functions of their inputs, a theoretical advantage over the linear neurons in DNNs, yet state-of-the-art WNN architectures still lag behind DNNs in accuracy on common classification tasks. Additionally, many existing WNN architectures suffer from high memory requirements, hindering implementation. In this paper, we propose a novel WNN architecture, BTHOWeN, with key algorithmic and architectural improvements over prior work, namely counting Bloom filters, hardware-friendly hashing, and Gaussian-based nonlinear thermometer encodings. These enhancements improve model accuracy while reducing size and energy per inference. BTHOWeN targets the large and growing edge computing sector by providing superior latency and energy efficiency to both prior WNNs and comparable quantized DNNs. Compared to state-of-the-art WNNs across nine classification datasets, BTHOWeN on average reduces error by more than 40% and model size by more than 50%. We demonstrate the viability of a hardware implementation of BTHOWeN by presenting an FPGA-based inference accelerator, and compare its latency and resource usage against similarly accurate quantized DNN inference accelerators, including multi-layer perceptron (MLP) and convolutional models. The proposed BTHOWeN models consume almost 80% less energy than the MLP models, with nearly 85% reduction in latency. In our quest for efficient ML on the edge, WNNs are clearly deserving of additional attention.

Weightless neural networks (WNNs) are an alternative pattern recognition technique where RAM nodes function as neurons. As both training and inference require mostly table lookups, few additions, and no multiplications, WNNs are suitable for high-performance and low-power embedded applications. This work introduces a novel approach to implement WiSARD, the leading WNN state-of-the-art architecture, completely eliminating memories and arithmetic circuits and utilizing only logic functions. The approach creates compressed minimized implementations by converting trained WNN nodes from lookup tables to logic functions. The proposed LogicWiSARD is implemented in FPGA and ASIC technologies to illustrate its suitability for edge inference. Experimental results show more than 80% reduction in energy consumption when the proposed LogicWiSARD model is compared with a multilayer perceptron network (MLP) of equivalent accuracy. Compared to previous work on FPGA implementations for WNNs, convolutional neural networks, and binary neural networks, the energy savings of LogicWiSARD range between 32.2% and 99.6%.

The binary encoding of real valued inputs is a crucial part of Weightless Neural Networks.The Linear Thermometer and its variations are the most prominent methods to determine binary encoding for input data but, as they make assumptions about the input distribution, the resulting encoding is sub-optimal and possibly wasteful when the assumption is incorrect.We propose a new thermometer approach that doesn't require such assumptions.Our results show that it achieves similar or better accuracy when compared to a thermometer that correctly assumes the distribution, and accuracy gains up to 26.3% when other thermometer representations assume an unsound distribution.

Weightless neural networks (WNNs) are a type of machine learning model which perform prediction using lookup tables (LUTs) instead of arithmetic operations.Recent advancements in WNNs have reduced model sizes and improved accuracies, reducing the gap in accuracy with deep neural networks (DNNs).Modern DNNs leverage "pruning" techniques to reduce model size, but this has not previously been explored for WNNs.We propose a WNN pruning strategy based on identifying and culling the LUTs which contribute least to overall model accuracy.We demonstrate an average 40% reduction in model size with at most 1% reduction in accuracy.

Conditional branch prediction is a technique used to speculatively execute instructions before knowing the direction of conditional branch statements. Perceptron-based predictors have been extensively studied, however, they need large input sizes for the data to be linearly separable. To learn nonlinear functions from the inputs, we propose a conditional branch predictor based on the WiSARD model and compare it with two state-of-the-art predictors, the TAGE-SC-L and the Multiperspective Perceptron. We show that the WiSARD-based predictor with a smaller input size outperforms the perceptron-based predictor by about 0.09% and achieves similar accuracy to that of TAGE-SC-L.

This paper proposes a weightless architecture for graph classification scenarios.This architecture is a three-headed arrangement composed of graph hand-picked features, a quantization method and a final classifier.Although multiple new strategies for graph classification have been proposed in recent years, it is still necessary to settle comparable studies with respect to weightless neural networks.The proposed architecture is evaluated along with other baseline classifiers and independent strategies, showing that weightless architectures are able to compete with other well-established methods such as graph kernels.* This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior -Brasil (CAPES) -Finance Code 001, CNPq, FAPERJ and DIPPG -CE-FET/RJ.

n-tuple neural networks have been in the past applied to a wide range of learning domains.However, for the particular area of regression, existing systems have displayed two shortcomings: little flexibility in the objective function being optimized and an inability to handle nonstationarity in an online learning setting.A novel n-tuple system is proposed to address these issues.The new architecture leverages the idea of functional gradient descent, drawing inspiration from its use in kernel methods.Furthermore, its capabilities are showcased in two reinforcement learning tasks, which involves both nonstationary online learning and task-specific objective functions.

This work takes into account the benefits of machine learning in order to estimate the valence of emotions on the OMG Empathy dataset, considering the information obtained from face expressions and dialogue of interlocutors. RegressionWiSARD and ClusRegressionWiSARD n-tuple regressors and its ensembles were employed to this end. The best performance achieved among all the combinations of weightless neural models considered (evaluated using the CCC metric) was 0.25 in validation set of the Personalized Track.

The growing volume of data produced continuously in the Cloud and at the Edge poses significant challenges for large-scale AI applications to extract and learn useful information from the data in a timely and efficient way. The goal of this article is to explore the use of computational storage to address such challenges by distributed near-data processing. We describe Newport, a high-performance and energy-efficient computational storage developed for realizing the full potential of in-storage processing. To the best of our knowledge, Newport is the first commodity SSD that can be configured to run a server-like operating system, greatly minimizing the effort for creating and maintaining applications running inside the storage. We analyze the benefits of using Newport by running complex AI applications such as image similarity search and object tracking on a large visual dataset. The results demonstrate that data-intensive AI workloads can be efficiently parallelized and offloaded, even to a small set of Newport drives with significant performance gains and energy savings. In addition, we introduce a comprehensive taxonomy of existing computational storage solutions together with a realistic cost analysis for high-volume production, giving a good big picture of the economic feasibility of the computational storage technology.

The Wilkie, Stonham, and Aleksander recognition device (WiSARD) [Formula: see text]-tuple classifier is a multiclass weightless neural network capable of learning a given pattern in a single step. Its architecture is determined by the number of classes it should discriminate. A target class is represented by a structure called a discriminator, which is composed of [Formula: see text] RAM nodes, each of them addressed by an [Formula: see text]-tuple. Previous studies were carried out in order to mitigate an important problem of the WiSARD [Formula: see text]-tuple classifier: having its RAM nodes saturated when trained by a large data set. Finding the VC dimension of the WiSARD [Formula: see text]-tuple classifier was one of those studies. Although no exact value was found, tight bounds were discovered. Later, the bleaching technique was proposed as a means to avoid saturation. Recent empirical results with the bleaching extension showed that the WiSARD [Formula: see text]-tuple classifier can achieve high accuracies with low variance in a great range of tasks. Theoretical studies had not been conducted with that extension previously. This work presents the exact VC dimension of the basic two-class WiSARD [Formula: see text]-tuple classifier, which is linearly proportional to the number of RAM nodes belonging to a discriminator, and exponentially to their addressing tuple length, precisely [Formula: see text]. The exact VC dimension of the bleaching extension to the WiSARD [Formula: see text]-tuple classifier, whose value is the same as that of the basic model, is also produced. Such a result confirms that the bleaching technique is indeed an enhancement to the basic WiSARD [Formula: see text]-tuple classifier as it does no harm to the generalization capability of the original paradigm.

High frequency trading depends on quick reactions to meaningful information. In order to identify opportunities in intraday negotiation in the stock markets, we propose a weightless neural network autonomous trader agent composed by forecasting and decision modules. The forecasting module uses ridge regression, which compared favorably against recursive least squares with exponential forgetting. The decision model applies the predicted prices to compute technical indicators based on a set of relative strength indicators evaluated by back-testing, which are then used to train the weightless neural network WiSARD in deciding whether to buy or sell stocks. Experimental results on a real dataset from the Brazilian stock market showed that it is feasible encode the back-testing in WiSARD in order to improve trading rules in a way that is compatible with the reaction time required by online market updates.

Music tracking is a useful technique for many music related tasks. Some applications include evaluating musicians performance, automatic score page turning and syncing music lyrics. In this work, we describe a WiSARD-based system for real-time music tracking and evaluate the performance of the corresponding implementation. Given any audio example, the system is capable of recognizing which point of the original signal is currently playing and of dynamically tracking it. In other words, if the music restarts or jumps to another position, the system is capable of following it. This is accomplished thanks to the low complexity training and classifying of the models used, which continually keep track of all possible points of the music, and not only of the current neighboring region. Experiments are provided in order to analyze the performance of the system in three test scenarios: tracking continuous playing, tracking multiple jumps and self predicting errors. The final results demonstrate that the system is efficient even when applied to musics with some level of repetitions and short periods of silence.

In this work, a generalized central pattern generator (CPG) model is formulated to generate a full range of gait patterns for a hexapod insect. To this end, a recurrent neural network module, as the building block for rhythmic patterns, is proposed to extend the concept of oscillatory building blocks (OBB) for constructing a CPG model. The model is able to make transitions between different gait patterns by simply adjusting one model parameter. Simulation results are further presented to show the effectiveness and performance of the CPG network.

This paper presents the implementation of ARQ-PROP II, a limited-depth propositional neural reasoner based on the Resolution Principle. The SATyrus platform was used in the synthesis of Energy functions from a set of pseudo-Boolean constraints specifying ARQ-PROP II architectures for different inferencing depths. Global minima of the Energy functions produced by SATyrus are associated to SATisfiability of a formula and, in the case of ARQ-PROP II, are associated to Resolution-based refutations. This allows for simplified abduction, prediction and planning to be unified with deduction in a goal-driven style, i.e. there is no need for presetting a reasoning style upon a target set of clauses. Experimental results on deduction with ARQ-PROP II using different propositional depth settings are presented together with a correction of Gadi Pinkas' mapping of SATisfiability into Energy minima.

This paper introduces SATyrus, a neuro-symbolic architecture oriented to optimization problem solving via mapping problems specification into sets of pseudo-Boolean constraints. SATyrus provides a logical declarative language used to specify and compile a target problem into a particular energy function representing its space state of solutions. The resulting energy function is then mapped into a higher-order Hopfield network of stochastic neurons in order to find its global minima. The application of SATyrus over three illustrative problems are given: (i) graph coloring, (ii) traveling salesperson problem (TSP), and (iii) calculus of the difference between observed and hypothesized distances of two atoms, a sub-problem of the determination of a molecular structure.

This work presents ARQ-PROP-II, the propositional version of a neural engine for finding proofs by refutation using the Resolution Principle. This neural architecture does not require special arrangements or modules to do forward or backward reasoning, being driven by the goal posed to it. ARQ-PROP-II is capable of integrated monotonic reasoning with complete and incomplete knowledge. The neural mechanism presented herein is the first to our knowledge that does not require that the knowledge base be either pre-encoded or learnt.

Abstract Neural networks, or connectionist systems, have recently emerged as a powerful model of collective, parallel computation of great interest in artificial intelligence and combinatorial optimization. The understanding of neural networks is still largely dependent upon simulations, which in turn can be of great interest to the designer of parallel software, owing to the inherently distributed character of those systems. This paper is concerned with the simulation of one specific class of neural networks, namely those introduced by J. J. Hopfield. We discuss the design and occam implementation of a distributed parallel simulator of such networks, allowing for both binary‐ and continuous‐response neurons. A design is provided which we judge to be generic to a large extent, and then problems related to an occam implementation are discussed. One problem of particular relevance is the potential occurrence of communication deadlocks as a result of the unbuffered communication among occam processes.