To achieve collaboration among multiple services, accurately understanding quality of service (QoS) data becomes an important task since the known QoS data varies over time and tends to be extremely sparse, which makes missing QoS data analysis a challenge. Latent Factorization of Tensor (LFT) has shown its potential to capture the pattern of service interactions effectively. However, the existing work has two shortcomings, i.e., a) most of them use simple feature spaces to represent QoS data, which cannot achieve global intermodal correlation, b) the representation ability of LFT models is restricted by the commonly used first-order optimizer due to their bilinear and nonconvex nature. To address the above issues, this work innovatively proposes a Second-order LFT Network (STN) It innovatively uses a fully-connected latent factor space to construct the feature space of each modality of QoS data and employs the principle of Hessian-free optimization for integrating second-order information. Experimental results on the Response-Time industrial QoS dataset show that STN achieves better understanding of QoS data for obtaining higher QoS prediction accuracy than its peers with affordable computational burden.

The frame-based method is not suitable for counting repetitive actions in event stream, since the framing process will disrupt the temporal information of events. For accurate count of repetitive actions in events, we propose a framework based on threefold ideas: a) converting event stream into time series, b) searching candidates of repetitive actions based on the ascending and descending trends of event time series, and c) checking the candidates with a fast dynamic time warping based method. For accurate counting repetitive actions, an action enhancement method for event time series and a Mann-Kendall test incorporated dynamic candidate selection algorithm are innovatively proposed. The experimental results on artificially synthesized and normally recorded event datasets demonstrate that our framework can count repetitive actions in event stream with high accuracy. All codes, datasets and examples of visualization can be found at https://github.com/ZYL618/action_count_in_events3.

Combinatorial optimization problems prevail in engineering and industry. Some are NP-hard and thus become difficult to solve on edge devices due to limited power and computing resources. Quadratic Unconstrained Binary Optimization (QUBO) problem is a valuable emerging model that can formulate numerous combinatorial problems, such as Max-Cut, traveling salesman problems, and graphic coloring. QUBO model also reconciles with two emerging computation models, quantum computing and neuromorphic computing, which can potentially boost the speed and energy efficiency in solving combinatorial problems. In this work, we design a neuromorphic QUBO solver composed of a swarm of spiking neural networks (SNN) that conduct a population-based meta-heuristic search for solutions. The proposed model can achieve about x20 40 speedup on large QUBO problems in terms of time steps compared to a traditional neural network solver. As a codesign, we evaluate the neuromorphic swarm solver on a 40nm 25mW Resistive RAM (RRAM) Compute-in-Memory (CIM) SoC with a 2.25MB RRAM-based accelerator and an embedded Cortex M3 core. The collaborative SNN swarm can fully exploit the specialty of CIM accelerator in matrix and vector multiplications. Compared to previous works, such an algorithm-hardware synergized solver exhibits advantageous speed and energy efficiency for edge devices.

Event and frame cameras capture the complemen-tary spatial and temporal details of a scene providing an accuracy vs. latency trade-off. Fusing these processing modalities using convolutional (CNN) and spiking neural networks (SNN) respectively has been shown for target tracking. We present our heterogeneous RRAM compute-in-memory (CIM) and SRAM compute-near-memory (CNM) SoC for simultaneous processing of CNN and SNN. We will show the advantage of using fused vision over frame-only vision and demonstrate python programmable data streaming. The visitors will be able to see the processing-dependent dynamic power gating of non-volatile RRAM and in-memory error correction capability.

The remarkable progress in artificial intelligence (AI) has ushered in a new era characterized by models with billions of parameters, enabling extraordinary capabilities across diverse domains. However, these achievements come at a significant cost in terms of memory and energy consumption. The growing demand for computational resources raises grand challenges for the sustainable development of energy-efficient AI systems. This paper delves into the paradigm of memory-based computing as a promising avenue to address these challenges. By capitalizing on the inherent characteristics of memory and its efficient utilization, memory-based computing offers a novel approach to enhance AI performance while reducing the associated energy costs. Our paper systematically analyzes the multifaceted aspects of this paradigm, highlighting its potential benefits and outlining the challenges it poses. Through an exploration of various methodologies, architectures, and algorithms, we elucidate the intricate interplay between memory utilization, computational efficiency, and AI model complexity. Furthermore, we review the evolving area of hardware and software solutions for memory-based computing, underscoring their implications for achieving energy-efficient AI systems. As AI continues its rapid evolution, identifying the key challenges and insights presented in this paper serve as a foundational guide for researchers striving to navigate the complex field of memory-based computing and its pivotal role in shaping the future of energy-efficient AI.

Learning how to walk is a sophisticated neurological task for most animals. In order to walk, the brain must synthesize multiple cortices, neural circuits, and diverse sensory inputs. Some animals, like humans, imitate surrounding individuals to speed up their learning. When humans watch their peers, visual data is processed through a visual cortex in the brain. This complex problem of imitation-based learning forms associations between visual data and muscle actuation through Central Pattern Generation (CPG). Reproducing this imitation phenomenon on low power, energy-constrained robots that are learning to walk remains challenging and unexplored. We propose a bio-inspired feed-forward approach based on neuromorphic computing and event-based vision to address the gait imitation problem. The proposed method trains a "student" hexapod to walk by watching an "expert" hexapod moving its legs. The student processes the flow of Dynamic Vision Sensor (DVS) data with a one-layer Spiking Neural Network (SNN). The SNN of the student successfully imitates the expert within a small convergence time of ten iterations and exhibits energy efficiency at the sub-microjoule level.

Online learning in legged robot under stringent performance and energy constraints thwarts the application of conventional reinforcement learning and optimization algorithms. The integration of complex sensors and data pre-processing required in using these algorithms makes this more challenging. Spiking neural networks allow local learning and low computing power opening new possibilities neuromorphic paradigm to such tasks. Central pattern generation based learning to walk in hexapod robots perfectly matches the temporal learning in SNNs allowing end-to-end learning. We propose a stochastic reinforcement-based algorithm allowing the hexapod to learn using the reward generated by the gyro sensors and camera-based visual inputs. The system is implemented on a Raspberry pi to demonstrate convergence to bio-observed gait patterns.

Building oscillator-based computing systems with emerging nano-device technologies has become a promising solution for unconventional computing tasks like computer vision and pattern recognition. However, simulation and analysis of these computing systems is both time and compute intensive due to the nonlinearity of new devices and the complex behavior of coupled oscillators. In order to speed up the simulation of coupled oscillator systems, we propose a simplified phase model to perform phase and frequency synchronization prediction based on a synthesis of earlier models. Our model can predict the frequency-locking behavior with several orders of magnitude speedup compared to direct evaluation, enabling the effective and efficient simulation of the large numbers of oscillators required for practical computing systems. We demonstrate the oscillator-based computing paradigm with three applications, pattern matching, convolution, and image segmentation. The simulation with these models are respectively sped up by factors of 780, 300, and 1120 in our tests.

Weakly coupled oscillators have shown promise as a computational building block that exploits the power of emerging low power high density nano-devices such as spin torque oscillators and vanadium oxide oscillators. In this paper, we develop a new analytic phase model as well as a circuit simulation to show how clusters and arrays of coupled oscillators can be inserted into three different stages of an image processing pipeline and provide comparable image recognition performance to traditional methods.

We discuss the opportunities of performing associative processing based on the phase locking of coupled oscillators. The use of coupled oscillators, rather than Boolean logic, provides for implementations using emerging nano-technology such as magnetic spin torque oscillators and resonant body transistor oscillators, which each have the potential of lower energy operations and higher density scaling than traditional CMOS solutions.