Event cameras have recently been introduced into image semantic segmentation, owing to their high temporal resolution and other advantageous properties. However, existing event-based semantic segmentation methods often fail to fully exploit the complementary information provided by frames and events, resulting in complex training strategies and increased computational costs. To address these challenges, we propose an efficient hybrid framework for image semantic segmentation, comprising a Spiking Neural Network branch for events and an Artificial Neural Network branch for frames. Specifically, we introduce three specialized modules to facilitate the interaction between these two branches: the Adaptive Temporal Weighting (ATW) Injector, the Event-Driven Sparse (EDS) Injector, and the Channel Selection Fusion (CSF) module. The ATW Injector dynamically integrates temporal features from event data into frame features, enhancing segmentation accuracy by leveraging critical dynamic temporal information. The EDS Injector effectively combines sparse event data with rich frame features, ensuring precise temporal and spatial information alignment. The CSF module selectively merges these features to optimize segmentation performance. Experimental results demonstrate that our framework not only achieves state-of-the-art accuracy across the DDD17-Seg, DSEC-Semantic, and M3ED-Semantic datasets but also significantly reduces energy consumption, achieving a 65% reduction on the DSEC-Semantic dataset.

In the realm of video object segmentation (VOS), the challenge of operating under low-light conditions persists, resulting in notably degraded image quality and compromised accuracy when comparing query and memory frames for similarity computation. Event cameras, characterized by their high dynamic range and ability to capture motion information of objects, offer promise in enhancing object visibility and aiding VOS methods under such low-light conditions. This paper introduces a pioneering framework tai-lored for low-light VOS, leveraging event camera data to elevate segmentation accuracy. Our approach hinges on two pivotal components: the Adaptive Cross-Modal Fusion (ACMF) module, aimed at extracting pertinent features while fusing image and event modalities to mitigate noise interference, and the Event-Guided Memory Matching (EGMM) module, designed to rectify the issue of in-accurate matching prevalent in low-light settings. Additionally, we present the creation of a synthetic LLE-DAVIS dataset and the curation of a real-world LLE-vas dataset, encompassing frames and events. Experimental evaluations corroborate the efficacy of our method across both datasets, affirming its effectiveness in low-light scenarios. The datasets are available at https://github.com/HebeiFast/EventLowLightVOS.

Clothes retrieval is very popular on e-commerce websites, and it is still a very challenging problem. These difficulties come from clothing deformation, variations in angle, noisy background and so on. According to our observation, although the overall shape of clothing varies greatly, there is slightly deformation in some local parts (such as collar, pocket, etc.) which has a significant impact on the final retrieval performance. In this paper, we propose an end-to-end attention-based deep network. It consists of two main branches, one of which is used to extract the key parts in clothing from different granularities. The other is designed to simulate the spatial contextual between different parts to obtain more discriminative feature representation by means of Long Short-Term Memory (LSTM). Experimental results demonstrate that our method is superior to previous methods in terms of performance and effectiveness.

We propose a novel method for multi-person 2D pose estimation. Our model zooms in the image gradually, which we refer to as the Magnify-Net, to solve the bottleneck problem of mean average precision (mAP) versus pixel error. Moreover, we squeeze the network efficiently by an inspired design that increases the mAP while saving the processing time. It is a simple, yet robust, bottom-up approach consisting of one stage. The architecture is designed to detect the part position and their association jointly via two branches of the same sequential prediction process, resulting in a remarkable performance and efficiency rise. Our method outcompetes the previous state-of-the-art results on the challenging COCO key-points task and MPII Multi-Person Dataset.

Clothes search is a challenging task, because the uncontrolled capture condition and massive styles of clothing images make the accurate retrieval difficult. To address the problem, we design a novel Attribute-and-Identity Correspondence Network (AICN) to obtain more robust features for clothes retrieval. The attribute information of clothes includes category, color, shape and so forth. The images of the same clothing pictured under different pose or background conditions have the same identity. Specifically, the AICN is comprised of two sub-branches: the attribute one and the identity one, and the two branches share the low level features. The former exploits attribute information to learn the discriminative feature, while the latter is to handle the variations of images with the same identity. Furthermore, we design a compactness loss to force the feature spaces of the two branches to become close, since their semantic spaces have overlap intuitively. Experiments on the benchmark and practical dataset demonstrate the effectiveness and priority of our method.