This work presents a high-speed 3D Gaussian Splatting-based SLAM (3DGS-SLAM) accelerator to support dense mapping for mobile devices. 3DGS-SLAM has two main hardware challenges for acceleration: 1) Large α-computation computation. 2) Memory bottleneck caused by irregular memory access and large number of Gaussians. First, diagonal feeding (DF) controller precludes redundant-α computation, and symmetric alpha reuse (SAR) enables reusing computed alpha. This method reduces 35.3% system computation. Second, voxel-based inter-frame caching (VIFC) enables selective inter-frame voxel caching, which reduces 44.0% of external memory access. As a result, the proposed 3DGS-SLAM accelerator achieves 51.2 fps with 0.07µJ/point with support voltage 0.9V, clock frequency 200MHz mapping a high-quality dense-map.

This work presents an energy-efficient and real-time rendering Neural-3DGS processor for mobile AR/VR devices. While Neural-3DGS shows high-quality and fast rendering, it exhibits low energy efficiency & high latency in mobile implementation. The proposed processor has three key features for the overall processes in Neural-3DGS: 1) Voxel Similarity-aware Memory Management Unit (VSMMU) eliminates redundant operations and achieves 59.5%, 43.3% reduced energy for external memory access and neural-network computation. 2) LUT-based Pre-Sort Unit (LPSU) utilizes pre-computed order to reduce the latency of sorting by 64.3%. 3) Opacity-aware Gaussian Skipping Core (OGSC) exploits sparsity based on opacity and process 63.3% reduced MAC operations. The proposed processor is implemented in 28 nm CMOS technology. It achieves 98.8 FPS for real-time rendering and 4.31 mJ/Frame energy efficiency.

An energy-efficient video understanding processor, SLYTHERIN, is proposed to accelerate the new AI model, mamba, efficiently on edge devices. Mamba, a state-of-the-art model for in-context learning, is designed to replace the transformer, whose computational complexity increases significantly in video applications. However, the acceleration of video mamba on edge devices presents two main challenges: 1) slow inference due to the iterative operation phase of mamba and 2) the large energy consumption caused by external memory access (EMA). To address these challenges, the SLYTHERIN is proposed with 3 key building blocks. 1) A 6-stage pipelined task allocator minimizes computational complexity by dynamically managing redundant computations with importance-aware prediction. 2) Reformed computing SSM engine increases core efficiency by tackling the overheads in mamba’s iterative stages with reordering and distributed L1 cache. 3) Patch data management unit addresses the large EMA with difference-based bit-sliced data compression. It finally achieves 2.67-mJ/frame system energy efficiency with 274 FPS.

•Why 3D Modeling using Neural Radiance Field?

Deep neural networks (DNNs) have achieved high performance in many AI fields such as 1-D language, 2-D image, and 3-D point cloud processing applications. Since recent DNN tasks require dense matrix operations with various bit-precision and non-ReLU activation functions, mobile neural processing units (NPUs) suffer from the acceleration of diverse DNN tasks within their limited hardware resources and power budget. Although bit-slice architectures benefit from slice-level computation and slice-level sparsity exploitation, the conventional bit-slice representation is inefficient in bit-slice architectures resulting in poor dense DNN execution. This paper proposes the efficient signed bit-slice architecture, Sibia, with the signed bit-slice representation (SBR) for efficient dense DNN acceleration. The SBR adds a sign bit to each bit-slice and changes signed 11112bit-slice to 00002by borrowing a value of 1 from its lower order of the bit-slice. This scheme generates large numbers of zero bit-slices in dense DNNs even not relying on accuracy-sensitive pruning methods or retraining processes. Moreover, the SBR balances positive and negative values of 2’s complement data, allowing accurate bit-slice-based output speculation that pre-computes high orders of bit-slices. Sibia integrates the signed multiplier-and-accumulate (MAC) units for efficient signed bit-slice computations, and the flexible zero skipping processing element (PE) supports the zero input bit-slice skipping and output skipping for high throughput and energy-efficiency. Additionally, the dynamic sparsity monitoring unit monitors sparsity ratio between input and weight data and determines the more sparse one for zero bit-slice skipping. The heterogeneous network-on-chip (NoC) benefits from data reusability during bit-slice computation, reducing transmission bandwidth. Finally, Sibia outperforms the previous bit-slice architecture, Bit-fusion, over 3.65× higher area-efficiency, 3.88× higher energy-efficiency, and 5.35× higher throughput.

A real-time dense 3D reconstruction on lightweight AR headsets is challenging since its memory access surpasses the available memory bandwidth. To solve this problem, the proposed processor integrates two key building blocks - Dilation-based TSDF (D-TSDF) fusion and Block-Projection (BP) engine. D-TSDF projects the depth map in the reverse order of voxel-to-pixel coordinate transformation and dilates it, leading to 96.61% External Memory Access (EMA) reduction with minimum map quality degradation. Second, a specialized BP engine compresses high-resolution occupancy grid by decomposing the 3D bitmap into 2D and 1D vectors, achieving$\times \mathbf{166.09}$reduced memory bandwidth. The proposed processor is implemented in 28nm CMOS technology occupying 1.27 mm2area. As a result, 96.45 fps 3D reconstruction is possible while consuming only 15.94 mW power.

This work presents an energy-efficient digital-based computing-in-memory (CIM) processor to support floating-point (FP) deep neural network (DNN) acceleration. Previous FP-CIM processors have two limitations. Processors with post-alignment shows low throughput due to serial operation, and the other processor with pre-alignment incurs truncation error. To resolve these problems, we focus on the statistics that outlier exists according to shift amount in pre-alignment-based FP operation. As those outlier decreases energy efficiency due to long operation cycles, it needs to be processed separately. The proposed Hetero-FP-CIM integrates both CIM arrays and shared NPU, so they compute both dense inlier and sparse outlier respectively. It also includes efficient weight caching system to avoid entire weight copy in shared NPU. The proposed Hetero-FP-CIM is simulated in 28 nm CMOS technology and occupies 2.7 mm2. As a result, it achieves 5.99 TOPS/W at ImageNet (ResNet50) with bfloat16 representation.

3D Data in Mobile Platforms