Event-based moving object detection is a challenging task, where static background and moving object are mixed together. Typically, existing methods mainly align the background events to the same spatial coordinate system via motion compensation to distinguish the moving object. However, they neglect the potential spatial tailing effect of moving object events caused by excessive motion, which may affect the structure integrity of the extracted moving object. We discover that the moving object has a complete columnar structure in the point cloud composed of motion-compensated events along the timestamp. Motivated by this, we propose a novel joint spatio-temporal reasoning method for event-based moving object detection. Specifically, we first compensate the motion of background events using inertial measurement unit. In spatial reasoning stage, we project the compensated events into the same image coordinate, discretize the timestamp of events to obtain a time image that can reflect the motion confidence, and further segment the moving object through adaptive threshold on the time image. In temporal reasoning stage, we construct the events into a point cloud along timestamp, and use RANSAC algorithm to extract the columnar shape in the cloud for peeling off the background. Finally, we fuse the results from the two reasoning stages to extract the final moving object region. This joint spatio-temporal reasoning framework can effectively detect the moving object from motion confidence and geometric structure. Moreover, we conduct extensive experiments on various datasets to verify that the proposed method can improve the moving object detection accuracy by 13%.

In this work, we propose a flexible and efficient instance segmentation framework, termed DISF (Dynamic Instance Segmentation with Semantic Features), which is a more novel two-stage instance segmentation framework. Firstly, we divide the image into multiple regions of the same size and directly classify the pixels in different regions, which converts the instance-level segmentation task into the pixel-level classification task within the region. We make full use of the location and size information of objects to distinguish different instances of the same category and obtain relatively coarse instance segmentation results. Secondly, we decouple the prediction of instance masks into convolution kernel prediction and instance features prediction. The instance masks are dynamically generated by convolution operations between the predicted convolution kernel and instance features. The segmentation results in this way do not contain redundant information. Thirdly, a parallel branch of semantic segmentation is added to refine instance segmentation results further. Semantic features provide global information about the image from a higher level. Semantic features and instance features are sent to the Features Fusion Module (FFM) to optimize the relatively coarse instance segmentation results generated in the previous stage. The experimental results reveal the promising potential of DISF in instance-level recognition.

Transformers, which are famous for their attention mechanisms, have a strong ability to extract global information. The advantage of CNNs is parameter sharing, which can extract local information well. Both global information and local information are important for pose recognition. However, the existing human pose estimation methods based on Transformers or their variants can not extract the local information of images very well in mid-sized datasets (See [1] for more details). Therefore, we propose a novel Transformer framework to solve the problem of human pose estimation, termed ATPose (Aggregation Transformer for Human Pose Estimation): (1) We embed the convolution operations into the decoder of the Transformer to extract the local information. The attention module of Transformer first extracts global information from feature maps, and then the convolution layers perform convolution operations on feature maps to focus on local information. Thus, the Transformer can extract global information and local information at the same time. (2) We introduce the sparse attention mechanism and multi-scale attention mechanism into the Transformer. The sparse attention mechanism allows the feature pixels to only interact with sampled pixels, rather than all pixels, which reduces the computation cost. Multi-scale attention mechanism can calculate attention on feature maps with different resolutions to better extract small target features. (3) The Keypoint Head module is added to the decoder. The Keypoint Head can refine the prediction results of the model, making the predicted coordinates more accurate, and can also guide the training to prevent the model from deviating. The experimental results show ATPose has achieved state-of-the-art performance and become a new baseline in regression-based methods.