Dynamic vision sensors (DVSs), renowned for their low latency and sparse event-driven output, have garnered significant attention in machine vision applications, particularly in latency-sensitive applications like object tracking. However, current DVS-based object tracking systems have not fully utilized the advantages of DVS due to their frame-based processing or the high computing intensity of their algorithms. This brief presents EDMOT, a fully event-driven multiobject tracking system for DVS, achieving 25–65 ns latency at 200 MHz. EDMOT introduces three key innovations: 1) a novel event-driven update mechanism that processes only the latest event and the expired oldest event, minimizing computational overhead; 2) a dual-threshold tracking strategy that decouples object formation and motion phases, significantly improving tracking accuracy; and 3) a row–column feature memory with flag registers, enabling object separation within eight clock cycles. The proposed EDMOT is evaluated on public datasets, demonstrating superior tracking accuracy compared to prior methods. Finally, EDMOT was implemented at the HLMC 55 nm, supporting 20–100 Me/s throughput. To the best of our knowledge, this is a multiobject tracking system with minimal delay and the highest event processing throughput.

This paper proposed an algorithm-hardware co-design of an event-driven spiking neural network (SNN) accelerator for classification tasks of event-based data from dynamic vision sensors (DVS), which can implement a feed-forward SNN with a maximum network size of 1 million synapses. Configurable structured sparsity is introduced between the first layer and the second layer to improve energy efficiency and balance the workload between different processing elements (PEs). The number of available neurons in the accelerator is sparsity-dependent and ranges from 1024 to 4096. The modified leaky-integrate-fire (LIF) neuron model and an event-driven neuron update scheme are employed in both algorithm and hardware to fully utilize the natural sparsity of DVS event stream. Early-stop inference strategy on the hardware enables a trade-off between inference accuracy and efficiency. A three-layer fully connected SNN is trained through backpropagation through time (BPTT) and is implemented and evaluated on Xilinx ZCU104 FPGA. Our design can achieve 96.0% accuracy on the N-MNIST dataset and 79.0% accuracy on the DVS128-Gesture dataset both at 50% sparsity. The top performance of the accelerator on ZCU104 is 3.22 GSOP/S and 3.99 GSOP/W at 250 MHz.

The defective pixel corrector is an essential component of the image processor, which detects and corrects defective pixels in the image. Processing discrete defective pixels that exist in the output of area-scan image sensors is the focus of current research. However, the case of defective columns produced by line scan image sensors is a massive challenge for existing algorithms. In this paper, we develop novel algorithms for the detection and correction of columnar defective pixels by modeling the properties of defective columns in the output images of line scan image sensors. In addition, we design the non-extremum verdict and texture adaptive correction for clustered defective pixels in the area scan sensors and apply them to current algorithms to get enhancement, and new algorithms are obtained. Moreover, we propose a generalized hardware architecture for detection and correction. In the experimental stage, the proposed methods are compared with the state-of-the-art methods, both at the algorithmic level and in hardware implementation. The experimental results show that, compared to the current widely used algorithms, our methods for line-scan image sensors improve the detection rate by 28%, the detection precision by 90%, and the image restoration quality by 35% with a 10% increase in hardware consumption. For area-scan image sensors, the proposed non-extremum verdict improves the detection rate by 25%, improves the precision from 0.056 to 0.983, and the texture adaptive correction mitigates the image blurring caused by the correction process, with a 37% improvement in image quality, while incurring only a 1% increase in hardware overhead. In addition, our proposed algorithms achieve comparable detection and correction results with far fewer computational and storage resource requirements than machine learning-based algorithms.

This brief designs and implements a 4-D imaging light detection and ranging (LiDAR) receiver. It employs a reconfigurable transimpedance amplifier (TIA) that alternates between two modes to separately achieve ranging and light intensity quantification functions. A new mode-switching method based on a monostable multivibrator is proposed, allowing the TIA to automatically switch modes during measurement. The reconfigurable TIA and mode-switching method enable the application of charge sampling in 4-D LiDAR imaging receiver, resulting in a higher signal-to-noise ratio (SNR) compared with traditional designs. In addition, the TIA mode used for distance measurement achieves a bandwidth of 140 MHz, a gain of$99.8~\text {dB}\Omega $, and an input-referred noise of ~20-nA rms, indicating high detection sensitivity. A prototype implemented in 0.18-um CMOS verifies the feasibility of the proposed receiver, consuming 24.5 mW. Measurement results demonstrate that the prototype can effectively acquire distance and light intensity information of target objects within a range of 5 m.

This paper proposed a hardware-algorithm co-design of an event-driven Spiking Neural Network (SNN) accelerator with structured sparsity for Dynamic Vision Sensors (DVS) applications. The accelerator can accommodate up to 1024 neurons and 1 million synapses for a feed-forward fully connected SNN implementation. Configurable structured sparsity is introduced by modular arithmetic both in the algorithm and hardware to improve the energy efficiency, reduce the memory requirement, and balance the workload between different processing elements. With an event-driven neuron update scheme, the accelerator can fully utilize the benefits of structured sparsity and can directly process DVS output data for classification tasks without encoding. A three-layer SNN is trained through backpropagation through time (BPTT) and is implemented on Xilinx ZCU104 FPGA, which achieves 96% accuracy on the N-MNIST dataset and 79% accuracy on the DVS-Gesture dataset both at 50% sparsity. The top performance of the accelerator on ZCU104 is 3.82 GSOP/S and 5.31 GSOP/W at 250 MHz.

Nowadays, with the increase resolution of Dynamic Vision Sensor (DVS), efficient compression algorithm for event stream is needed urgently. Conventional DVS system encodes event data in address event representation (AER) for output while ignores the data redundancy imposed by the correlation of events. To address this challenge, this paper first analyzes the spatiotemporal characteristics of event stream and the impact of readout circuits. Based on the analysis, the context-based encoding strategies for spatial address, timestamp and polarity of events are proposed respectively with the consideration of data flow in DVS hardware. Besides, the hardware architecture with high parallelism is presented to implement the compression algorithm, which achieves high throughput at an affordable cost. The hardware is implemented in the 55nm process as part of a 512x512 resolution DVS. The experimental results demonstrate that our methods achieves higher average compression ratio compared to conventional and DVS-specific coding algorithms.

This article proposes a multimode neuromorphic event-frame integrated vision sensor that enables event detection (ED) with simultaneous brightness measurement based on the pulse width modulation mechanism. The logarithmic voltage is directly taken as intensity information. Brightness measurement involves in-pixel voltage-to-pulse conversion and out-pixel pulse coding. The maximum event bandwidth is improved to 366 Meps by pipelining the time-prior arbiter along with the address-events grouping circuit. A wide intensity dynamic range of 105 dB can theoretically be achieved through logarithmic photoelectric conversion and pulse coding. Our sensor supports a$128\times 64$frame-like image with an improved signal-to-noise ratio of 49 dB. The experimental results indicate that the log sensitivity of the optimized logarithmic photoreceptor was measured as 164 mV/dec. The equivalent frame rate for both event and intensity reaches kilo fps, making it a promising candidate in high-speed wireless sensing applications.

The Dynamic Vision Sensor (DVS) is a new type of bionic vision image sensor that offers the advantages of low latency, low power consumption, and high dynamics range compared to conventional sensors. However, background activity (BA) noise will degrade the quality of the DVS output data and lead to unnecessary bandwidth overhead. In dark environments, pixel arrays generate abundant noise, and conventional spatiotemporal filters can hardly achieve satisfactory results. To solve this problem, we exploit the difference in event density distribution between the actual event and noise and propose a denoising method that utilizes the event densities with two neighbors of different radii. Compared to spatiotemporal filters, our approach reduces the error rate on synthetic datasets by at least 35%. Meanwhile, our approach is subjectively more visually appealing. With our denoising method, the performance of DVS can be better in dark conditions.

Dynamic vision sensors (DVS) have significant potential in scenes involving high-speed motion and extreme light. However, DVS is sensitive to background active noise, which will degrade the quality of the output. The ordinary$O(N^{2})$-Space spatiotemporal filter's memory complexity is high. It needs$N\times N$memory cells ($N\times N$is the resolution on the sensor). Some works reduce memory complexity by sacrificing the performance of the filter. To ensure the filtering effect and reduce the filter's memory complexity, this paper proposes a novel filter: Queue-based spatiotemporal filter. Moreover, based on the Queue-based spatiotemporal filter, this paper proposes a clustering algorithm that can cluster while filtering. Experiments show that the proposed filter's performance is similar to the$O(N^{2})$-Space spatiotemporal filter while having a lower memory complexity. Besides, using the proposed clustering algorithm, the objects in motion can be clustered with low calculation complexity.

Multitemporal interferometric synthetic aperture radar (InSAR) technology has become an important tool for digital elevation model (DEM) generation and surface deformation monitoring. The prerequisite is to obtain the correct unwrapped phase of the interferograms. However, it is still challenging to obtain reliable results by traditional 2-D phase unwrapping (PU) algorithms and even 3-D PU algorithms, when the phase continuity assumption fails due to large deformation gradients or abrupt topographic changes. This letter proposes an${L}^{ {1}}$-norm sparse 2-D PU algorithm based on reliable redundant network in the phase discontinuous region, which is obtained by implementing ambiguity detector on free network within a given distance. Experiment on real synthetic aperture radar (SAR) interferograms covering urban tall buildings verified that the algorithm can obtain more reliable results in phase discontinuous regions than traditional PU algorithms.

Surface shortwave net radiation (SSNR) and surface downward shortwave radiation (DSR) are the two surface shortwave radiation components in earth's radiation budget and the fundamental quantities of energy available at the earth's surface. Although several global radiation products from global circulation models, global reanalyses, and satellite observations have been released, their coarse spatial resolutions and low accuracies limit their application. In this paper, the Global LAnd Surface Satellite (GLASS) DSR product was generated from the Moderate Resolution Imaging Spectroradiometer top-of-atmosphere (TOA) spectral reflectance based on a direct-estimation method. First, the TOA reflectances were derived based on the atmospheric radiative transfer simulations under different solar/view geometries; second, a linear regression relationship between the TOA reflectance and SSNR was developed under various atmospheric conditions and surface properties for different solar/view geometries; third, the coefficients derived from the linear regression were used to compute the SSNR; and finally, the DSR was estimated using the SSNR estimates and broadband albedo at the surface. A 13-year (2003-2015) GLASS DSR product was generated at a 5-km spatial resolution and 1-day temporal resolution. Compared with the ground measurements collected from 525 stations from 2003 to 2005 around the world, the model-computed SSNR (DSR) had an overall bias of 8.82 (3.72) W/m2and a root mean square error of 28.83 (32.84) W/m2at the daily time scale. Moreover, the global land annual mean of the DSR was determined to be 184.8 W/m2with a standard deviation of 0.8 W/m2over a 13-year (2003-2015) period.