Privacy-Preserving Image Retrieval (PPIR) enables searching for similar images on cloud servers while safeguarding image privacy. PPIR uploads encrypted images to servers to address privacy concerns and then employs deep neural networks for retrieval on extracted features from cipher-images. However, current PPIR encrypts all images with the same secret keys to maintain consistent feature spaces, lacking support for adaptive keys where distinct images are encrypted with various keys. To this end, we propose a new PPIR scheme to support adaptive keys while keeping stable feature spaces. Specifically, we design ingenious image encryption to align with feature extraction during the JPEG compression process, incorporating encryptable orthogonal transformations, shuffling, stream cipher, and sign encryption operations. Our approach extracts well-designed absolute value sequences of local blocks and global histogram features from cipher-images, ensuring stable feature spaces under adaptive keys. To enhance model generalization performance, we employ Self-Supervised Contrastive Learning (SSCL) to build a pretraining model and propose a straightforward yet efficient block-sampling augmentation technique for the structured features to drive SSCL. Moreover, our retrieval model implements a dual-attention structure to capture dependencies among local block sequences and import scores of global features. Extensive experiments on four datasets demonstrate that our approach achieves superior retrieval accuracy compared to existing schemes and maintains excellent retrieval performance under adaptive keys, effectively preserving image privacy.

Image retrieval systems help users to browse and search among extensive images in real time. With the rise of cloud computing, retrieval tasks are usually outsourced to cloud servers. However, the cloud scenario brings a daunting challenge of privacy protection as cloud servers cannot be fully trusted. To this end, image-encryption-based privacy-preserving image retrieval (PPIR) schemes have been developed, which first extract features from cipher-images, and then build retrieval models based on these features. Yet, most existing PPIR approaches extract shallow features and design trivial unsupervised retrieval models, resulting in insufficient expressiveness for the cipher-images. In this paper, we propose a novel paradigm named Encrypted Vision Transformer (EViT), which advances the discriminative representations capability of cipher-images. First, to capture comprehensive ruled information, we extract multi-level local length sequence and global Huffman-Code frequency features from the cipher-images which are encrypted by permutation encryption, sign encryption, and stream cipher during the JPEG compression process. Second, we design the modified self-supervised Vision Transformer with Huffman-embedding and propose two robust data augmentations on cipher-images to improve representation power of the retrieval model. Moreover, our proposal can be easily adapted to unsupervised or supervised settings. Extensive experiments reveal that EViT achieves both excellent encryption and retrieval performance, outperforming current schemes in terms of retrieval accuracy by large margins while protecting image privacy effectively. Code is publicly available at https://github.com/onlinehuazai/EViT.

In recent years, people tend to upload images to cloud servers, which provide storage and retrieval functions. To prevent users’ privacy from leaking to the server, research on cipher-image retrieval has attracted much attention. This work presents a novel encrypted image retrieval method. With this scheme, we perform encryption during the JPEG compression process by applying 16×16 DCT (Discrete Cosine Transform) for blocks’ transformation, followed by coefficients distribution and 8×8 blocks’ permutation. For the retrieval part, when an encrypted query image is sent by an authorized user, the server extracts its DCT histograms as features and inputs them into our trained network model, which incorporates transpose Multilayer perceptron modules ($Transpose$$MLP$), for retrieval. Experimental results show that our scheme, compared with related schemes, can improve the retrieval performance significantly, when ensuring compression friendliness and no feature information leakage. Moreover, our scheme enables cipher-image retrieval from multiple image owners.

Applying encryption technology to image retrieval can ensure the security and privacy of personal images. The related researches in this field have focused on the organic combination of encryption algorithm and artificial feature extraction. Many existing encrypted image retrieval schemes cannot prevent feature leakage and file size increase or cannot achieve satisfied retrieval performance. In this paper, a new end-to-end encrypted image retrieval scheme is presented. First, images are encrypted by using block rotation, new orthogonal transforms and block permutation during the JPEG compression process. Second, we combine the triplet loss and the cross entropy loss to train a network model, which contains gMLP modules, by end-to-end learning for extracting cipher-images' features. Compared with manual features extraction such as extracting color histogram, the end-to-end mechanism can economize on manpower. Experimental results show that our scheme has good retrieval performance, while can ensure compression friendly and no feature leakage.