Graph-based multi-task learning at billion-scale presents a significant challenge, as different tasks correspond to distinct billion-scale graphs. Traditional multi-task learning methods often neglect these graph structures, relying solely on individual user and item embeddings. However, disregarding graph structures overlooks substantial potential for improving performance. In this paper, we introduce the Macro Graph of Experts (MGOE) framework, the first approach capable of leveraging macro graph embeddings to capture task-specific macro features while modeling the correlations between task-specific experts. Specifically, we propose the concept of a Macro Graph Bottom, which, for the first time, enables multi-task learning models to incorporate graph information effectively. We design the Macro Prediction Tower to dynamically integrate macro knowledge across tasks. MGOE has been deployed at scale, powering multi-task learning for a leading billion-scale recommender system, Alibaba. Extensive offline experiments conducted on three public benchmark datasets demonstrate its superiority over state-of-the-art multi-task learning methods, establishing MGOE as a breakthrough in multi-task graph-based recommendation. Furthermore, online A/B tests confirm the superiority of MGOE in billion-scale recommender systems.

Human Gesture Recognition (HGR) using mmWave radar has become increasingly promising due to its exceptional contactless perception sensitivity. Conventional approaches predominantly rely on supervised models to learn radar signals, thus incurring substantial costs associated with annotation. To address this limitation, certain works embrace transfer learning to effectively transfer knowledge from labeled source domain to unlabeled target domain, achieving unsupervised recognition in the target domain. However, existing transfer-based methods still necessitate large-scale labeled source domain radar data, thereby constraining their practical applicability. To this end, we propose a novel unsupervised solution for mmWave-based HGR by transferring public image gestures to radar data, eliminating the need for acquiring labeled radar data in source domain. We aim to establish heterogeneous alignment between images and radar signals, facilitating cross-domain transfer. Initially, we mitigate the negative impact of data heterogeneity by employing sophisticated signal processing techniques to convert raw radar signals into gesture trajectories. Subsequently, we introduce an Adversarial-Contrastive Domain Transfer Model (ACDTM) to achieve fine-grained alignment. ACDTM not only confuses the source and target domains by adversarial learning, enabling the acquisition of domain-invariant features, but also designs a robust similarity matrix to facilitate intra-class alignment through contrastive learning. Additionally, ACDTM conducts adversarial self-training on target domain with pseudo-labeled distribution. Our experimental findings substantiate that the unsupervised accuracy achieves about 80$\sim$92% on different mmWave gesture datasets, outperforming existing unsupervised HGR schemes by large margins. Code is available athttps://github.com/onlinehuazai/mmGesture.

WiFi sensing advancements facilitate the capture of human gestures from wireless signals, ensuring both privacy preservation and robustness under low-light conditions. Deep learning-based WiFi Human Gesture Recognition (HGR) demonstrates remarkable performance in handling complex gestures. To reduce labeling efforts, recent years have seen the emergence of semi-supervised WiFi HGR, leveraging massive amounts of unlabeled data. However, existing semi-supervised schemes often assume a balanced class distribution and utilize a fixed threshold for selecting pseudo-labels of unlabeled samples, leading to low performance for minority classes and decreased model generalization on real-world imbalanced datasets. To address this issue, we propose a novel semi-supervised WiFi HGR approach with dynamic pseudo-labeling thresholds to handle imbalanced class distribution, incorporating Spatial-Temporal Attention (STA) networks. Unlike using a fixed threshold for all unlabeled samples, our design implements class-independent thresholds for different classes, dynamically adjusting them by encoding pseudo-label distribution during training. To emphasize critical features in informative areas within the WiFi signals, we incorporate both spatial self-attention and temporal attention mechanisms to dynamically learn salient features and identify pivotal frames, respectively. Moreover, we introduce adaptive WiFi data augmentations that propel the semi-supervised framework and enhance model robustness. Experimental results on the Widar3.0 dataset reveal that our approach outperforms existing semi-supervised methods by large margins in accuracy, effectively mitigating imbalanced bias and enhancing model generalization.

To meet the new requirements of Industry 4.0, the logistics field has introduced 3D reconstruction technology. Computer vision-based solutions face challenges like bad lighting conditions and line-of-sight constraints. Meanwhile, the widespread adoption of RFID tags in supply chains offers an opportunity to enhance current reconstruction methods. In this article, we propose TagRecon, a fine-grained multi-object 3D reconstruction scheme utilizing well-deployed RFIDs. Specifically, TagRecon transforms the task of reconstruction into a problem of estimating 3D bounding boxes for tagged packages. By placing dual anchor tags on each target package, TagRecon enables accurate inference of the package’s translation and rotation using RFID-based localization and orientation sensing. Our scheme introduces a novel method to estimate rotations and translations for tagged packages, utilizing the known geometric relationship of anchor tags. Besides, to achieve simultaneous reconstruction of multiple packages, we manage to match tags from various packages through the correlation between anchor tag pairs. As far as we know, this is the first RFID-based solution that can simultaneously realize 3D translation and rotation estimation of multiple objects to a fine granularity. Experiments validate TagRecon achieves a 28.0 cm translation error and 6.8°, 6.0°, and 7.5° rotation errors for roll, pitch, and yaw angles on average.

Privacy-Preserving Image Retrieval (PPIR) enables searching for similar images on cloud servers while safeguarding image privacy. PPIR uploads encrypted images to servers to address privacy concerns and then employs deep neural networks for retrieval on extracted features from cipher-images. However, current PPIR encrypts all images with the same secret keys to maintain consistent feature spaces, lacking support for adaptive keys where distinct images are encrypted with various keys. To this end, we propose a new PPIR scheme to support adaptive keys while keeping stable feature spaces. Specifically, we design ingenious image encryption to align with feature extraction during the JPEG compression process, incorporating encryptable orthogonal transformations, shuffling, stream cipher, and sign encryption operations. Our approach extracts well-designed absolute value sequences of local blocks and global histogram features from cipher-images, ensuring stable feature spaces under adaptive keys. To enhance model generalization performance, we employ Self-Supervised Contrastive Learning (SSCL) to build a pretraining model and propose a straightforward yet efficient block-sampling augmentation technique for the structured features to drive SSCL. Moreover, our retrieval model implements a dual-attention structure to capture dependencies among local block sequences and import scores of global features. Extensive experiments on four datasets demonstrate that our approach achieves superior retrieval accuracy compared to existing schemes and maintains excellent retrieval performance under adaptive keys, effectively preserving image privacy.

Image retrieval systems help users to browse and search among extensive images in real time. With the rise of cloud computing, retrieval tasks are usually outsourced to cloud servers. However, the cloud scenario brings a daunting challenge of privacy protection as cloud servers cannot be fully trusted. To this end, image-encryption-based privacy-preserving image retrieval (PPIR) schemes have been developed, which first extract features from cipher-images, and then build retrieval models based on these features. Yet, most existing PPIR approaches extract shallow features and design trivial unsupervised retrieval models, resulting in insufficient expressiveness for the cipher-images. In this paper, we propose a novel paradigm named Encrypted Vision Transformer (EViT), which advances the discriminative representations capability of cipher-images. First, to capture comprehensive ruled information, we extract multi-level local length sequence and global Huffman-Code frequency features from the cipher-images which are encrypted by permutation encryption, sign encryption, and stream cipher during the JPEG compression process. Second, we design the modified self-supervised Vision Transformer with Huffman-embedding and propose two robust data augmentations on cipher-images to improve representation power of the retrieval model. Moreover, our proposal can be easily adapted to unsupervised or supervised settings. Extensive experiments reveal that EViT achieves both excellent encryption and retrieval performance, outperforming current schemes in terms of retrieval accuracy by large margins while protecting image privacy effectively. Code is publicly available at https://github.com/onlinehuazai/EViT.

In recent years, people tend to upload images to cloud servers, which provide storage and retrieval functions. To prevent users’ privacy from leaking to the server, research on cipher-image retrieval has attracted much attention. This work presents a novel encrypted image retrieval method. With this scheme, we perform encryption during the JPEG compression process by applying 16×16 DCT (Discrete Cosine Transform) for blocks’ transformation, followed by coefficients distribution and 8×8 blocks’ permutation. For the retrieval part, when an encrypted query image is sent by an authorized user, the server extracts its DCT histograms as features and inputs them into our trained network model, which incorporates transpose Multilayer perceptron modules ($Transpose$$MLP$), for retrieval. Experimental results show that our scheme, compared with related schemes, can improve the retrieval performance significantly, when ensuring compression friendliness and no feature information leakage. Moreover, our scheme enables cipher-image retrieval from multiple image owners.

Applying encryption technology to image retrieval can ensure the security and privacy of personal images. The related researches in this field have focused on the organic combination of encryption algorithm and artificial feature extraction. Many existing encrypted image retrieval schemes cannot prevent feature leakage and file size increase or cannot achieve satisfied retrieval performance. In this paper, a new end-to-end encrypted image retrieval scheme is presented. First, images are encrypted by using block rotation, new orthogonal transforms and block permutation during the JPEG compression process. Second, we combine the triplet loss and the cross entropy loss to train a network model, which contains gMLP modules, by end-to-end learning for extracting cipher-images' features. Compared with manual features extraction such as extracting color histogram, the end-to-end mechanism can economize on manpower. Experimental results show that our scheme has good retrieval performance, while can ensure compression friendly and no feature leakage.