The rapid growth of virtual reality (VR), augmented reality (AR), extended reality (XR), and intelligent surveillance systems has driven increasing demand for video understanding tasks on edge devices. However, supporting such tasks on edge devices remains challenging due to the limited computational resources, memory constraints, power budgets, and high data transition latency. To address these challenges, this work proposes an ultralow-power smart vision System-on-Chip (SoC) with the following features: 1) a bitline segmented, parallel-charging read-only memory (ROM) compute-in-memory (CIM) macro with power gating and coupled coding; 2) heterogeneous similarity-aware hybrid SRAM/ROM cores to exploit temporal similarity with high core utilization; 3) an intracore and intercore pipeline with hierarchical dataflow for low-power data transmission; 4) a tilewise mixed-precision weight quantization and mapping scheme for weight data compression; and 5) a hierarchical multimodal system trigger mechanism utilizing an on-chip CMOS imager with near sensor caching to skip unnecessary inferences. The proposed ROM CIM macro achieves an area efficiency of 0.753–1.673 TOPS/mm2and a storage density of 4706 Kb/mm2, achieving a$3.2\!\!-\!\!14.83\times $improvement in density Figure of Merit (FoM) over the state-of-the-art (SoTA) ROM CIM designs. The proposed SoC also shows an ultralow always-on power of$0.13~\mu $W and an average active power of 9.52 mW, which demonstrates a high energy efficiency of 13.6–18.3 TOPS/W (ResNet-20 with fixed-point 8-bit activation and 10-bit weight) on video understanding tasks, showing a$1.8-3.0\times $improvement over the existing smart vision SoCs.

The proliferation of augmented reality (AR), virtual reality (VR), and extended reality (XR) edge devices drives the demand for video understanding, which imposes substantial demands on computational resources, memory, and energy efficiency. This work proposes an ultra-low power and highly compact smart vision SoC, composed of: 1) a bit-line (BL) segmented parallel charging ROM CIM macro with power gating and coupled coding; 2) heterogeneous similarity-aware hybrid SRAM/ROM cores for exploiting temporal similarity with high utilization; 3) a hierarchical multi-modal system trigger uses an on-chip CMOS imager with near-sensor caching for skipping unnecessary inference; and 4) a RISC-V core featuring dedicated ISA extensions to support flexible workload allocation and programmability. The proposed ROM CIM macro achieves a storage density of 4705 kb/mm2and an area efficiency of 0.753 TOPS/mm2, demonstrating a 3.2~14.83× improvement in density FoM compared to state-of-the-art (SOTA) CIM macros. Furthermore, the SoC achieves an energy efficiency of 13.6~18.3 TOPS/W on ResNet-20 (fixed-point 8-bit activation and 10-bit weight), representing a 1.8~3.0× improvement over existing smart vision SoCs.

Always-on intelligent visual perception applications are widely deployed in edges in the AIoT era. In order to eliminate power costs of data conversion and transmission, this paper proposes Senputing, an ultra-low-power processing-in-sensor chip that completely fuses sensing and computing together for a BNN-based hierarchical processing system. This chip could operate in two modes. In computation mode, photocurrents are directly utilized for computing without being converted into voltages, and the computation results of 1-st BNN layer are directly sent out to subsequent BNN processors for an always-on coarse classification, eliminating conversion power and storage cost of raw images. Once an interested objected is detected, this chip switches to sensor mode and sends raw images to potential full-precision processors or cloud servers for fine-grained recognition or segmentation. A$32\times 32$prototype is fabricated with 180nm CMOS process. It accomplishes MNIST dataset classification task with the accuracy of 93.76% and the power consumption of 147nW at 156fps, achieving$13.1\times $energy efficiency compared with state-of-the-art work.