Real-time ego-motion tracking and 3D structural estimation are the fundamental tasks for the ubiquitous cyper-physical systems, and they can be conducted via the state-of-the-art Edge-Based Visual Odometry (EBVO) algorithm. However, the intrinsic data-intensive process of EBVO emplaces a memory-wall hurdle in practical deployment on conventional von-Neumann-style computing systems. In this work, we attempt to leverage SRAM based processing-in-memory (PIM) technique to alleviate such memory-wall bottleneck, so as to optimize the EBVO systematically from the perspectives of the algorithm layer and physical layer. In the algorithm layer, we first investigate the data reuse patterns of the essential computing kernels required for the feature detection and pose estimation steps in EBVO, and propose PIM friendly data layout and computing scheme for each kernel accordingly. We distill the basic logical and arithmetical operations required in the algorithm layer, and in the physical layer, we propose a novel bit-parallel and reconfigurable SRAM-PIM architecture to realize the operations with high computing precision and throughput. Our experimental result shows that the proposed multi-layer optimization allows for high tracking accuracy of EBVO, and it can improve 11x processing speed and reduce 20x energy consumption compared to the CPU implementation.

Ego-motion estimation with 3D perception using visual odometry (VO) is known to be robust and economical among the existing odometry techniques. However, existing VO solutions are typically both computation intensive and memory intensive, which dramatically inhibits their deployment in IoT platforms such as robotic vehicles and handheld devices mostly equipped with resource-constrained MCU-level processors. To enable real-time and high-quality VO on these scenarios with thrifty resource budgets, we investigate state-of-the-art edge-based VO (EBVO) and propose an optimization framework called PicoVO that can greatly reduce the amount of computation as well as the memory footprint from the perspectives of both algorithm and implementation. First of all, we revisit the key processing stages of EBVO and propose an EBVO-oriented lightweight edge detector in the pre-processing stage, a sparse-to-dense processing scheme in the tracking stage, and a lightweight key-frame management in the post-processing stage. In addition to the algorithmic optimization, we further develop a dedicated quantization scheme particularly for the 3D feature calculation and Levenberg-Marquardt (LM) solver that are critical to the memory footprint and computation requirements of PicoVO. Evaluation on realistic RGB-D benchmark datasets is conducted on NUCLEO-F767ZI equipped with a 216MHz Cortex-M7 MCU and 512KB RAM. It reveals that PicoVO achieves 33fps@320x240 with high tracking precision comparable to state-of-the-art VOs on PC.