This work proposes a self-supervised neuromorphic processor using high-dimensional representations for cognitive map navigation. By employing the Cognitive Map Learner (CML), it enables agents to explore and understand diverse environments online through random walks. To enhance path planning, the agent’s actions and observations are embedded into high-dimensional state spaces. This embedding creates a sense of direction, simplifying navigation into a retrieval process within an Associative Memory (AM). We design an energy-efficient processor that features a scalable multi-core hardware architecture with precision flexibility, combined with an on-chip random walk training engine. To balance the precision of the model with hardware overhead, two hardware-software co-design strategies are proposed. The first is a Content-Addressable Memory (CAM)-based approach for AM access, which reduces the number of memory access by up to 25%. The second involves high-dimensional matrix sparsity optimizations, reducing computation operations to less than 8%. We simulate this processor by a 40-nm CMOS technology, which has 2.88 mm2core area with 15.8 mW power at a frequency of 140 MHz. Compared to previous processors, our experiments show that the proposed processor achieves outstanding success rates of 99.9%, 96%, and 98.7% on 100 2D nodes, 125 3D nodes, and 25 abstract map nodes with obstacles, respectively. In terms of energy efficiency, it delivers a path planning result of 28 nJ/node and 35 nJ/node in 2D and 3D maps, offering a 1.2x to 2.9x improvement over the state-of-the-art.

This paper presents an efficient near-eye dynamic vision sensor (DVS)-based processor for real-time eye tracking in augmented reality/virtual reality (AR/VR) devices. The processor takes advantage of the sparse event data with fine time resolution from the DVS, addressing the need for high frame-rate, low-power, and accurate eye tracking on wearable devices with extended battery life. Exploiting the inherent sparsity of event data, we propose an event-density-based region of interest (ROI) determination method that operates directly on event stream, which requires$47\times $fewer operations than the traditional methods, effectively overcoming the latency problem caused by the heavy computational loads. To eliminate the issue of decreasing accuracy at the edges of the field of view (FoV), we customized and fine-tuned a neural network for gaze estimation, ensuring uniformly distributed sub-degree accuracy. An estimator with a streamlined output mapping strategy and an adaptive window-sliding convolution scheme is implemented for gaze estimation acceleration. The processor is designed and fabricated in UMC 40-nm LP technology with a core area of 2.52 mm2 and performs end-to-end eye tracking exclusively with the raw event stream from DVS, achieving an average accuracy of 0.91° within a$96^{\circ } \times 64^{\circ }$FoV. Operating at 200 MHz, it achieves a dynamic frame rate of up to 1.2 kHz and requires only$12.7~\mu $J of energy per gaze estimation. By integrating the DVS, the processor enables real-time, low-power, and accurate eye tracking, enhancing the immersive experience on AR/VR devices and offering intuitive and seamless interactions.

In this work, we present a neuromorphic processor for artificial intelligence of things (AIoT) applications featuring low-power consumption, a small footprint, STDP-based online learning, and the ability to adapt to multiple applications. Hybrid precision, i.e., INT8 for inference and INT16 for training, is suggested to achieve balanced accuracy and energy efficiency. A precision-configurable leaky integrate-and-fire(LIF) neuron unit and a unified memory architecture are designed to maximize datapath reuse. A dynamic pruning technique is proposed to exploit the temporal sparsity, yielding synaptic operations reduction by 3.68x in training and 1.63x in inference, respectively. The design is implemented and fabricated in a 40-nm CMOS process, with a core area of 0.87 mm2. It is measured to consume a minimal power of$680~\mu \text{W}$at 70 MHz under a 0.75 V power supply, corresponding to 9.9 pJ per synaptic operation. Evaluated with typical spatial, temporal, and spatiotemporal datasets (MNIST, MIT-BIH, and N-MNIST), the proposed design achieve energy efficiency comparable to the best-in-class solutions with handcrafted training and customized ASICs, while demonstrating improved versatility across multiple applications with balanced accuracy, power consumption, and model adaptability.