Recently, masked image modeling (MIM), which learns visual representations by reconstructing the masked patches of an image, has become a popular self-supervised paradigm. However, the pre-training of MIM always takes massive time due to the large-scale data and large-size backbones. We mainly attribute it to the random patch masking in previous MIM works, which fails to leverage the crucial semantic information for effective visual representation learning. To tackle this issue, we propose the Frequency & Attention-driven Masking and Throwing Strategy (FAMT), which can detect semantic patches and reduce the number of training patches to boost model performance and training efficiency simultaneously. Specifically, FAMT utilizes the self-attention mechanism to extract semantic information from the image for masking during training in an unsupervised manner. However, attention alone could sometimes focus on inappropriate areas regarding the semantic information. Thus, we are motivated to incorporate the information from the frequency domain into the self-attention mechanism to derive the sampling weights for masking, which captures semantic patches for visual representation learning. Furthermore, we introduce a patch throwing strategy based on the derived sampling weights to reduce the training cost. FAMT can be seamlessly integrated as a plug-and-play module and surpasses previous works, e.g. reducing the training phase time by nearly 50% and improving the linear probing accuracy of MAE by $1.8$ % ~ $ 6.3$ % across various datasets, including CIFAR-10/100, Tiny ImageNet, and ImageNet-1K. FAMT also demonstrates superior performance in downstream detection and segmentation tasks.

Wi-Fi signal-based person identification has become a hot research topic due to the widespread deployment of Wi-Fi devices and the fact that these approaches are noncontact, passive, and privacy-preserving. While the existing related methods and systems have achieved good performance for person identification, they also encounter many significant challenges in practical applications. Due to the propagation properties of Wi-Fi signals, the signal at the receiver will change significantly when the user’s appearance changes. This makes single-appearance trained models unusable for cross-appearance recognition tasks. To address this challenge, we propose a deep learning-based framework for appearance-independent identification using Wi-Fi signals (WiAi-ID), the core of which lies in the fact that the domain discriminator and feature extractor are trained together in an adversarial manner, thus forcing the model to extract identity-inherent features independent of human appearance, and introduces a multiscale CNN adaptation module to capture time-span-based features. We collected Wi-Fi signal data of pedestrians with different appearances. The experimental results show that WiAi-ID can effectively eliminate the impact on identification due to pedestrian appearance variations and accordingly outperforms the current state-of-the-art video and wireless signal-based recognition methods.

Most existing person re-identification (Re-ID) methods rely on high-cost manual annotations. To overcome the applicable issue, we focus on a novel semi-supervised Re-ID without cross-camera annotations, which we call random camera supervised person Re-ID (RCS). It is beneficial to real-world application, since a short-time and cheap annotation is conductive to rapid deployment of person re-ID. But only a small proportion of identities are annotated under a random camera, which is extremely challenging for Re-ID cross-camera pairing without cross-camera labeling or a labeled image for each identity. Towards reliable cross-camera learning, we propose a random camera guided framework (RCG) that can fully make use of the few labeled images with promising performance. RCG has two components: 1) Different from other complex methods to improve the clustering accuracy, Random camera guided clustering is adopted to mine cross-camera images of each identity, where the few labeled data helps to guide the simple but effective cluster split and combination. 2) Network learning under RCS is conducted with cluster-wise and camera-wise contrastive learning, where we deal with the camera variance in subgroup unit innovatively and further emphasis the importance of the labeled images. Extensive experiments on three large-scale Re-ID datasets show that our proposed approach not only outperforms state-of-the-art methods by a large margin, but achieves better performance with less annotation and more flexible RCS setting.

It has been witnessed that masked image modeling (MIM) has shown a huge potential in self-supervised learning in the past year. Benefiting from the universal backbone vision transformer, MIM learns self-supervised visual representations through masking a part of patches of the image while attempting to recover the missing pixels. Most previous works mask patches of the image randomly, which underutilizes the semantic information that is beneficial to visual representation learning. On the other hand, due to the large size of the backbone, most previous works have to spend much time on pre-training. In this paper, we propose Attention-driven Masking and Throwing Strategy (AMT), which could solve both problems above. We first leverage the self-attention mechanism to obtain the semantic information of the image during the training process automatically without using any supervised methods. Masking strategy can be guided by that information to mask areas selectively, which is helpful for representation learning. Moreover, a redundant patch throwing strategy is proposed, which makes learning more efficient. As a plug-and-play module for masked image modeling, AMT improves the linear probing accuracy of MAE by 2.9% ~ 5.9% on CIFAR-10/100, STL-10, Tiny ImageNet, and ImageNet-1K, and obtains an improved performance with respect to fine-tuning accuracy of MAE and SimMIM. Moreover, this design also achieves superior performance on downstream detection and segmentation tasks.