Event cameras have the potential to overcome the limitations of classical computer vision in real-world applications. Depth estimation is a crucial step for high-level robotics tasks and has attracted much attention from the community. In this paper, we propose an event-based dense depth estimation architecture, Mixed-EF2DNet, which firstly predicts inter-grid optical flow to compensate for lost temporal information, and then estimates multiple contextual depth maps that are fused to generate a robust depth estimation map. To supervise the network training, we further design a smoothing loss function used to smooth local depth estimates and facilitate estimating reasonable depth for pixels without events. In addition, we introduce SE-resblocks in the depth network to enhance the network representation by selecting feature channels. Experimental evaluations on both real-world and synthetic datasets show that our method performs better in terms of accuracy when compared to state-of-the-art algorithms, especially in scene detail estimation. Besides, our method demonstrates excellent generalization in cross-dataset tasks.

Event cameras are bio-inspired sensors that respond to pixel-level brightness changes in the form of asynchronous and sparse events. Each event is a 4-D tuple, i.e., (timestamp, x, y, polarity). Recently, algorithms for object detection based on learning have made considerable strides for event cameras. These methods transform event sequences into a tensor-like representation which can be processed by deep learning methods. However, these conversion methods don’t make full use of the polarity information of event sequences. We discover that tensor-like representations of different polarities show different features. Thus, we propose Polar Loss, a loss function that can enhance the difference between tensor-like representations of different polarities. To evaluate the effectiveness of our loss function, we design a network architecture for object detection. Our results state that our model shows good flexibility and expressiveness on event-based object detection when trained with the Polar Loss.

Recently, the emerging event camera shows great potential for robotics and AR/VR application thanks to its advantages including low latency, high dynamic range, low power consumption, etc. However, the output of event cameras usually has a large amount of noise which will affect unlocking their potentials and obtaining wider applications. In this paper, we present a two-stage local spatio-temporal event filter (LSTEF) based on adaptive thresholds. Considering the spatio-temporal constraints of generated event streams, a local sliding window is adopted for the noise candidate selection stage and noise filtering stage. An adaptive thresholding mechanism is also introduced into the filter in order to improve the generalization performance. Corresponding experimental evaluations are performed on the public datasets to prove the efficiency of the proposed filter. Results show that the presented LSTEF can successfully achieve the event denoising, and at the same time, effectively preserve useful scene information.

Compared with conventional cameras, dynamic vision sensors (namely event cameras) are especially suitable for high-speed and high-dynamic applications due to their advantages (high dynamic range, low latency, etc.). However, it still struggles when encountering low-speed and low-texture scenes. Aiming at the problem of optical flow estimation, we propose a novel unsupervised learning estimation method with both event data and gray image frames as the input. Two different fusion mechanisms are presented and discussed in our work, one is the Direct Fusion Network (DFEV-FlowNet), and the other one is Local Squeeze Extraction Network (LSENet). DFEV-FlowNet directly fuses synthesized event frames and gray image frames, and LSENet adopts a local squeeze extraction weights adaptive mechanism. In order to achieve self-supervised learning, photometric constraints between consecutive frames are used to drive the network training. We use the public event dataset MVSEC to evaluate the proposed optical flow estimation method qualitatively and quantitatively. The results show that our method demonstrate better performance in terms of the estimation accuracy.