This paper presents a flash LiDAR system based on successive approximation (SA) in-pixel histogramming time-to-digital converter (hTDC) featuring pulse position modulation (PPM) to suppress interference from other light sources. The pulse position is randomly shifted with the start time of the hTDC, spreading out interference equally to both bins and canceling it by the up-down counting with no demodulation process. The PPM modulator is implemented in an FPGA with a flash LiDAR sensor and is fully characterized. The proposed SA-based PPM successfully attenuates 16-fold larger interference than the user’s optical power, achieving a frame rate of 30 fps and a success rate of 99.9% at the same time.

As artificial intelligence (AI) technology advances, Internet of Things (IoT) devices, such as mobile phones and augmented reality devices, are increasingly becoming crucial enablers of user-device interactions. Among the various methods of interaction, hand pose recognition and analysis is a crucial method to understand the intentions of users and perform precise functions. However, to perform such functions, a substantial amount of computation and resources are required, making it challenging to implement them on small form-factor devices with low-power consumption. For this reason, improving energy efficiency is a crucial objective in real-time hand pose estimation (HPE) applied to low-power platforms with limited resources. In this article, we introduce an FPGA-based energy-efficient real-time HPE system with an integrated image signal processor (ISP). The proposed system uses several low-power design techniques, including a systolic array with dynamic on/off control per processing element (PE), to minimize power consumption and save energy when not in use. In addition, we improve area efficiency by reducing the buffer size in the systolic array using a half-size shift buffer stack. Furthermore, the use of parallel and pipelined structures improved operational efficiency, resulting in a reduction in both operational time and power consumption. The evaluation results on a KU115 FPGA board show that the system achieves an error of 7.78 mm and can process 52 fps, demonstrating its capability for real-time HPE. Moreover, this system achieves high-energy efficiency, up to 61.74 GOPs/W, making it suitable for energy-efficient and accurate HPE in low-power environments.

In-memory Computing (IMC) using Phase Change Memory (PCM) has proven to be effective for efficient processing of Deep Neural Networks (DNNs). However, with the use of multi-level cell PCM (MLC-PCM) in NVMs-based accelerators, errors due to resistance drift in MLC-PCM can severely degrade the DNNs accuracy. In this paper, an analysis of the impact of resistance drift errors on accuracy of MLC-PCM based DNN accelerator shows that the drift errors alone can significantly impact the accuracy. This paper proposes Hydra, which is a hybrid resistance drift resilient architecture for MLC-PCM based DNN accelerators which use IMC for efficient computations. Hydra utilizes Tri-level cell PCM, which has a negligible resistance drift error rate, to store the critical bits of DNNs parameters and MLC-PCM (4-level cell), which has a higher error rate (but offers more storage density), for the non-critical bits. Experimental results on various DNN architectures, configurations and datasets show that, with the presence of resistance drift errors in PCM, Hydra can maintain the baseline accuracy of DNNs for up to 1 year (resistance drift is time-dependent), whereas conventional drift tolerance techniques lead to a significant accuracy drop in just a few seconds.

Deep neural networks (DNNs) have emerged as the most effective programming paradigm for computer vision and natural language processing applications. With the rapid development of DNNs, efficient hardware architectures for deploying DNN-based applications on edge devices have been extensively studied. Emerging nonvolatile memories (NVMs), with their better scalability, nonvolatility, and good read performance, are found to be promising candidates for deploying DNNs. However, despite the promise, emerging NVMs often suffer from reliability issues, such as stuck-at faults, which decrease the chip yield/memory lifetime and severely impact the accuracy of DNNs. A stuck-at cell can be read but not reprogrammed, thus, stuck-at faults in NVMs may or may not result in errors depending on the data to be stored. By reducing the number of errors caused by stuck-at faults, the reliability of a DNN-based system can be enhanced. This article proposes CRAFT, i.e., criticality-aware fault-tolerance enhancement techniques to enhance the reliability of NVM-based DNNs in the presence of stuck-at faults. A data block remapping technique is used to reduce the impact of stuck-at faults on DNNs accuracy. Additionally, by performing bit-level criticality analysis on various DNNs, the critical-bit positions in network parameters that can significantly impact the accuracy are identified. Based on this analysis, we propose an encoding method which effectively swaps the critical bit positions with that of noncritical bits when more errors (due to stuck-at faults) are present in the critical bits. Experiments of CRAFT architecture with various DNN models indicate that the robustness of a DNN against stuck-at faults can be enhanced by up to$10^{5}$times on the CIFAR-10 dataset and up to 29 times on ImageNet dataset with only a minimal amount of storage overhead, i.e., 1.17%. Being orthogonal, CRAFT can be integrated with existing fault-tolerance schemes to further enhance the robustness of DNNs against stuck-at faults in NVMs.

We present a low-power indirect time-of-flight (iTOF) image sensor with fixed depth noise compensation and dual-mode imaging for depth dynamic range (DDR) enhancement. To reduce the power consumption from high-frequency pixel modulation, a TX driver with a single-sided clock chain is employed in the sensor. The inherent phase delay of the clock chain and the delay of the row bus are measured using row-parallel and column-parallel time-to-digital converters (TDCs) to compensate for the column and row fixed depth noise (FDN). To achieve a wide depth dynamic range (WDDR), the reconfigurable pixels and column circuits support dual-mode: short-range (SR) and long-range (LR) modes. A WDDR image is generated in a single frame through the mixed reconfiguration of the pixel array and interpolation. In addition, the temporal noise is suppressed without a significant time budget through a fast multiple sampling (FMS) scheme with 10b successive approximation register (SAR) analog-to-digital (ADCs). A prototype iTOF image sensor was fabricated using a 110 nm frontside illumination (FSI) CMOS image sensor (CIS) process and fully characterized. The sensor achieved a DDR of 4 m (0.7 to 4.7 m) with less than 1.7% nonlinearity and 0.9% depth noise. The FDN was suppressed to less than 2.1 cm at a low power consumption below 70 mW through the proposed compensation scheme using row and column TDCs. The temporal noise was only 0.48 mV$_{\mathbf {rms}}$owing to the FMS.

Deep Neural Networks (DNNs) have been found to outperform conventional programming approaches in several applications such as computer vision and natural language processing. Efficient hardware architectures for deploying DNNs on edge devices have been actively studied. Emerging memory technologies with their better scalability, non-volatility, and good read performance are ideal candidates for DNNs which are trained once and deployed over many devices. Emerging memories have also been used in DNNs accelerators for efficient computations of dot-product. However, due to immature manufacturing and limited cell endurance, emerging resistive memories often result in reliability issues like stuck-at faults, which reduce the chip yield and pose a challenge to the accuracy of DNNs. Depending on the state, stuck-at faults may or may not cause error. Fault-tolerance of DNNs can be enhanced by reducing the impact of errors resulting from the stuck-at faults. In this work, we introduce simple and light-weight Intra-block Address remapping and weight encoding techniques to improve the fault-tolerance for DNNs. The proposed schemes effectively work at the network deployment time while preserving the network organization and the original values of the parameters. Experimental results on state-of-the-art DNN models indicate that, with a small storage overhead of just 0.98%, the proposed techniques achieve up to 300× stuck-at faults tolerance capability on Cifar10 dataset and 125× on Imagenet datatset, compared to the baseline DNNs without any fault-tolerance method. By integrating with the existing schemes, the proposed schemes can further enhance the fault resilience of DNNs.