Simultaneous Localization and Mapping (SLAM) is a critical task that enables autonomous vehicles to construct maps and localize themselves in unknown environments. Recent breakthroughs combine SLAM with 3D Gaussian Splatting (3DGS) to achieve exceptional reconstruction fidelity. However, existing 3DGS-SLAM systems provide insufficient throughput due to the need for multiple training iterations per frame and the vast number of Gaussians.

3D Gaussian Splatting (3DGS) introduces a novel methodology for representing scenes with anisotropic 3D Gaussian primitives, achieving exceptional quality and rendering speed in neural scene representation (NSR). However, the insufficient training speed of 3DGS limits its applicability in tasks that require online learning to perceive dynamic environments, such as autonomous driving and embodied intelligence. Although recent work, GSCore, has introduced a specialized accelerator for the rendering process of 3DGS, it overlooks the time-consuming backward propagation during 3DGS training.In this paper, we propose GSArch, a hardware architecture designed to overcome memory barriers and boost the efficiency of 3DGS training. Through a thorough characterization of 3DGS training, we identify three root causes of inefficiency: redundant data loading from off-chip memory, time-consuming atomic write operations, and severe bank conflicts during on-chip buffer reading. To address these challenges, GSArch introduces three architectural innovations. First, acknowledging that Gaussians vary in shape and often span multiple pixels, with larger Gaussians causing more repetitive data loading, GSArch employs hybrid memory management. This approach categorizes Gaussians into ‘hot’ and ‘cold’ ones, storing hot Gaussians in a fast but small on-chip buffer to reduce redundant loading while minimizing hardware costs. Second, GSArch leverages the informativeness variability of Gaussians’ gradients to filter out low-contribution gradients, significantly reducing atomic operations. Lastly, a rearrangement unit is designed to pack conflicting memory read requests into non-conflicting bundles. Our evaluation results demonstrate that GSArch achieves up to $6.49 \times$ and $15.42 \times$ speedups compared to Nvidia A100 and Jetson AGX Xavier, respectively, with substantially lower energy consumption and negligible image quality loss.

Neural Radiance Field (NeRF) holds immense promise for generating photo-realistic images and videos. How-ever, the computation and memory demands significantly impede its applicability in real-time tasks such as virtual reality and augmented reality. While accelerators dedicatedly designed for NeRF have emerged, they may not be the optimal solution as they only consider the optimization of full-precision NeRF models, leading to unsatisfactory memory and computation saving. This paper proposes Sensitivity-Aware Dynamic Precision Rendering, abbreviated as SRender, an algorithm-hardware co-design framework that borrows the idea of Adaptive Rendering commonly used in conventional graphics rendering to accelerate the rendering process of NeRF exploiting the fine-grained per-ray and per-point levels sensitivity. The fundamental idea behind SRender is that rays and points with high sensitivity require high-precision data types, while the insensitive rays and points do not, allowing them to be sacrificed for greater acceleration. Accordingly, the algorithm part of SRender dynamically configures the precision of the hash table and the multilayer perceptron (MLP), and the hardware part features a specialized accelerator that exploits such mixed-precision memory access and computation patterns. Specifically, we predict the sensitive rays and points based on low-precision hash encoding and feature computation. Then, we trigger a high-precision rendering again on the sensitive ones to guarantee the rendering quality. The hardware design of SRender includes a sensitivity prediction engine, coarse-then-fine-grained encoding engines, and a recovery engine, which translates the computational saving in the algorithm to real speedup. Using the coarse-then-fine-grained encoding engine, we maximize the data reuse on-chip, thereby reducing the stress of off-chip memory bandwidth. Experiments show that the edge-level SRender can achieve 3.94x and 34.06x speedup over state-of-the-art NeRF accelerator NeuRex and Jetson AGX Xavier GPU with negligible rendering quality loss.