High dynamic range (HDR) imaging is essential for edge devices. Hybrid event-frame cameras can provide relative intensity events under HDR conditions and absolute intensity frames. Their image reconstruction pipeline needs to fuse different imaging data in real-time. Due to a lack of large-scale real-world datasets, deep learning algorithms can only rely on synthetic data for training, which face challenges with generalization ability. Model-based computational vision methods do not require training data. However, their event-driven update strategy results in slow processing speeds, making it difficult to achieve real-time performance on edge devices. In this paper, we present a fast HDR image reconstruction pipeline comprising Deblur, Calibration and Fusion, which respectively correspond to frame enhancement, decompression error calibration and nonlinear regression fusion. Our experiments are evaluated on public datasets with challenging lighting conditions. Our algorithm demonstrates improved performance metrics, achieving a 54% faster speed compared to state-of-the-art computational vision methods.

The development of lightweight technologies has made deploying convolutional neural networks on edge devices popular. However, the overflow caused by low-bit accumulators significantly degrades the accuracy of the model. Therefore, there is a challenge in balancing accuracy and low-bit accumulators. In this letter, we propose a novel method applying for training low-bit quantized neural network named Intermediate-Value Slip Quantization (ISQ). ISQ is used to constrain weights to decrease the risk of accumulator's overflow. Besides, we also set a criterion for ISQ to be aware of the bit width of the accumulator. In addition, we propose a method to integrate bias and Batch Normalization (BN) into the ISQ. This allows the computations to be shifted from the floating-point domain to the fixed-point. The experiment results demonstrate that ISQ effectively suppresses the overflow. The model accuracy under the 16-bit accumulator can be restored to 73.16% from 13.7% on CIFAR-100 with the same quantization configuration. Through our method, the design space of the hardware can be explored and the target accuracy can be achieved with the lowest hardware resources.

As millimeter-wave radar technology advances, modern domain-specific digital signal processors (DSPs) struggle to balance versatility and processing scale, while suffering from low data precision and data throughput. To address these challenges, this paper presents a novel coarse-grained versatile DSP (CVDSP). The CVDSP introduces an architecture based on multi-level finite state machines and a custom instruction set to support various algorithms through flexible dataflow. The efficient large-scale PEs are designed with 22-bit floating-point precision to handle 26K-point baseband signal processing and$2048\times 256$-point complex fast Fourier transform. Cooperating with a high-bandwidth instruction-free memory access network, the CVDSP achieves relatively high data throughput. Fabricated in a 65-nm CMOS process, experimental results show that the peak energy efficiency and data throughput of the CVDSP are 299.5 GMACs/J and 75.6 Gb/s. The CVDSP demonstrates fully on-chip implementation of the FMCW radar, Pulse-Doppler radar, and spectrometer algorithms.

Edge computing of reliable multimodal (2D RGB/3D RGB-Depth) data has a wide range of applications. However, many of currently reported visual processors cannot flexibly handle multimodal data, e.g., the visual streams of RGB-Depth data. The key challenge exists that these prior visual processors do not come with efficient and unified instruction set architecture (ISA) for both conventional and intelligent cognition on the 2D/3D multimodal sensory data. To fill such a gap, this paper proposes a programmable intelligent visual vector processor compatible with multimodal 2D/3D visual data processing ($1920\times 1080$-pixel resolution). The processor consists of a reconfigurable processing element (PE) array, a memory access network flexibly configurable to be fine- or coarse-grained, and a high throughput I/O interface. The vectorial PE array with neighbor PE access increases the data reuse rate and parallel computation efficiency, and can implement both convolutional neural networks (CNNs) and conventional image processing algorithms. The proposed ISA is customized and optimally tailored targeting 2D/3D image processing from RGB/Time-of-Flight(ToF) raw data to intelligent inference results. The chip is fabricated in a 55-nm CMOS process. The experimental results showed that the area efficiency, peak performance, and peak throughput of our chip attained as high as 14.41GOPS/mm2, 409.6GOPS, and 9.6Gbps at 200MHz, respectively. The measured processing speeds of this chip on ToF depth reconstruction is 87fps ($480\times 270$) or 31 fps($1920\times 1080$),on 3D object classification is 219fps ($256\times 256$), and on CNN-based 2D object tracking is 36fps ($256\times 256$).