Perceptual image encryption serves as a pivotal mechanism for delegating processing while ensuring the visual security of image data. The robustness of such encryption schemes is traditionally evaluated through cryptanalysis techniques, yet these approaches heavily rely on manual labor and prerequisite knowledge of the encryption algorithms. Recently, some works attempt to reveal visual content from perceptually encrypted images Based on CNN architectures. However, it is still tricky for these increasingly complex methods to reveal informative visual details. In this study, we focus on the extraction and utilization of inherent hierarchical features within the input image itself to significantly advances the field. To achieve it, we present a novel Progressive Fusion Attack Network (PFAN) to fully explore the hierarchical features. PFAN incorporates multiple subbranches, forming a progressive fusion structure that facilitates informative hierarchical feature representations and offers robust model fault tolerance. To enhance the reconstruction of encryption-induced distortions, we incorporate a Multiscale Feature Extraction Module (MFEM) that captures robust hierarchical features across various scales. Meanwhile, a Hierarchical Feature Fusion Module (HFFM) is designed to adaptively integrate and highlight the optimal feature representations, further optimizing the visual content reconstruction process. Extensive experimental evaluation demonstrates that PFAN exhibits remarkable agnosticism towards different perceptual encryption schemes and encryption strengths, achieving superior performance. Furthermore, PFAN outperforms state-of-the-art CNN-based image restoration methods in terms of effectiveness and generalizability.

Assessing the stealthiness of adversarial perturbations is challenging due to the lack of appropriate evaluation metrics. Existing evaluation metrics, e.g.,$L_{p}$norms or Image Quality Assessment (IQA), fall short of assessing the pixel-level stealthiness of subtle adversarial perturbations since these metrics are primarily designed for traditional distortions. To bridge this gap, we present the first comprehensive study on the subjective and objective assessment of the stealthiness of adversarial perturbations from a visual perspective at a pixel level. Specifically, we propose new subjective assessment criteria for human observers to score adversarial stealthiness in a fine-grained manner. Then, we create a large-scale adversarial example dataset comprising 10586 pairs of clean and adversarial samples encompassing twelve state-of-the-art adversarial attacks. To obtain the subjective scores according to the proposed criterion, we recruit 60 human observers, and each adversarial example is evaluated by at least 15 observers. The mean opinion score of each adversarial example is utilized for labeling. Finally, we develop a three-stage objective scoring model that mimics human scoring habits to predict adversarial perturbation’s stealthiness. Experimental results demonstrate that our objective model exhibits superior consistency with the human visual system, surpassing commonly employed metrics like PSNR and SSIM.

Backdoor attacks pose a serious security threat to Natural Language Processing (NLP) models, allowing adversaries to manipulate model outputs through hidden triggers. Although backdoor detection methods have been developed to address this issue, existing approaches based on trigger inversion are effective only for simple, visible triggers. These methods struggle to handle semantically enhanced, invisible triggers and often fail to provide accurate backdoor determinations due to reliance on unreliable heuristics, making it difficult to reliably distinguish backdoored models from benign ones. This presents a critical gap in current detection techniques. To address these challenges, we propose a novel trigger inversionSemInvthat consists of two key contributions: consistent semantics inversion and identifiable condition inspection. Consistent semantics inversion introduces a new regularization technique into the trigger optimization process, enabling more effective inversion of semantically constrained triggers. Identifiable condition inspection assesses the attack performance margin across different identifiable conditions, providing robust evidence for distinguishing backdoored models from benign ones. We evaluateSemInvusing the TrojAI round 6–8 datasets and demonstrate that it significantly outperforms state-of-the-art approaches in both backdoor detection accuracy and trigger inversion performance. Our method also proves effective against models with stealthy triggers, advancing the field of NLP security by offering a more comprehensive solution for identifying backdoor attacks. The code repository is in https://github.com/Bluedask/SemInv.

Perceptual image encryption degrades image quality by selectively encrypting some key information of the plain images. The encrypted images are partially perceptible according to the security or quality requirements. Although several types of attacks have tried to infer privacy information from the encrypted images, they can only either extract statistical information or enhance image sketch. In this paper, we take one step further and fully recover the plain images from perceptually encrypted counterparts by designing a non-local attack network (NL-ANet). NL-ANet is composed of densely cascaded multiscale non-local modules (MSNL) and a hierarchical attention fusion module (HAFM). In particular, to better reconstruct encryption distortion, we introduce MSNL to capture powerful hierarchical features from different scales, and propose HAFM to adaptively aggregate and enhance informative hierarchical features for reconstruction. We also propose a new instantiation of the multi-head non-local block with channel attention (MHCA) to explore the long-range dependencies of global contextual information. Extensive experiments show that NL-ANet is encryption-agnostic and superior on different perceptual encryption schemes under different encryption strengths. NL-ANet also achieves better performance than state-of-the-art image restoration methods.

Diffusion-weighted imaging (DWI) has been extensively explored in guiding the clinic management of patients with breast cancer. However, due to the limited resolution, accurately characterizing tumors using DWI and the corresponding apparent diffusion coefficient (ADC) is still a challenging problem. In this paper, we aim to address the issue of super-resolution (SR) of ADC images and evaluate the clinical utility of SR-ADC images through radiomics analysis. To this end, we propose a novel double transformer-based network (DTformer) to enhance the resolution of ADC images. More specifically, we propose a symmetric U-shaped encoder-decoder network with two different types of transformer blocks, named as UTNet, to extract deep features for super-resolution. The basic backbone of UTNet is composed of a locally-enhanced Swin transformer block (LeSwin-T) and a convolutional transformer block (Conv-T), which are responsible for capturing long-range dependencies and local spatial information, respectively. Additionally, we introduce a residual upsampling network (RUpNet) to expand image resolution by leveraging initial residual information from the original low-resolution (LR) images. Extensive experiments show that DTformer achieves superior SR performance. Moreover, radiomics analysis reveals that improving the resolution of ADC images is beneficial for tumor characteristic prediction, such as histological grade and human epidermal growth factor receptor 2 (HER2) status.

The Retinex model is one of the most representative and effective methods for low-light image enhancement. However, the Retinex model does not explicitly tackle the noise problem and shows unsatisfactory enhancing results. In recent years, due to the excellent performance, deep learning models have been widely used in low-light image enhancement. However, these methods have two limitations. First, the desirable performance can only be achieved by deep learning when a large number of labeled data are available. However, it is not easy to curate massive low-/normal-light paired data. Second, deep learning is notoriously a black-box model. It is difficult to explain their inner working mechanism and understand their behaviors. In this article, using a sequential Retinex decomposition strategy, we design a plug-and-play framework based on the Retinex theory for simultaneous image enhancement and noise removal. Meanwhile, we develop a convolutional neural network-based (CNN-based) denoiser into our proposed plug-and-play framework to generate a reflectance component. The final image is enhanced by integrating the illumination and reflectance with gamma correction. The proposed plug-and-play framework can facilitate both post hoc and ad hoc interpretability. Extensive experiments on different datasets demonstrate that our framework outcompetes the state-of-the-art methods in both image enhancement and denoising.

Vision-based practical applications, such as consumer photography and automated driving systems, greatly rely on enhancing the visibility of images captured in night-time environments. For this reason, various image enhancement algorithms (EHAs) have been proposed. However, little attention has been given to the quality evaluation of enhanced night-time images. In this paper, we conduct the first dedicated exploration of the subjective and objective quality evaluation of enhanced night-time images. First, we build an enhanced night-time image quality (EHNQ) database, which is the largest of its kind so far. It includes 1,500 enhanced images generated from 100 real night-time images using 15 different EHAs. Subsequently, we perform a subjective quality evaluation and obtain subjective quality scores on the EHNQ database. Thereafter, we present an objective blind quality index for enhanced night-time images (BEHN). Enhanced night-time images usually suffer from inappropriate brightness and contrast, deformed structure, and unnatural colorfulness. In BEHN, we capture perceptual features that are highly relevant to these three types of corruptions, and we design an ensemble training strategy to map the extracted features into the quality score. Finally, we conduct extensive experiments on EHNQ and EAQA databases. The experimental and analysis results validate the performance of the proposed BEHN compared with the state-of-the-art approaches. Our EHNQ database is publicly available for download athttps://sites.google.com/site/xiangtaooo/.

Research on image dehazing has made the need for a suitable dehazed image quality assessment (DIQA) method even more urgent. The performance of existing DIQA methods heavily relies on handcrafted haze-related features. Since hazy images with uneven haze density distributions will result in uneven quality distributions after dehazing, the manually extracted feature expression is neither accurate nor robust. In this paper, we design a deep CNN-based DIQA method without a handcrafted feature requirement. Specifically, we propose a blind dehazed image quality assessment model (BDQM), which consists of three components: image preprocessing, a haze-related feature extraction network (HFNet), and an improved regression network (IRNet). In HFNet, we design a perceptual information enhancement (PIE) module to learn powerful feature representations and enhance network capability according to channel attention, multiscale convolution and residual concatenation. IRNet aims to aggregate all patch information for the quality prediction of the whole image, where the effect of inhomogeneous distortion from the dehazing procedure is attenuated via a specifically designed patch attention (PA) mechanism. Experimental results on benchmark datasets demonstrate the effectiveness and superiority of the proposed network architecture over state-of-the-art methods.

Perceptual encryption is an efficient way of protecting image content by only selectively encrypting a portion of significant data in plain images. Existing security analysis of perceptual encryption usually resorts to traditional cryptanalysis techniques, which require heavy manual work and strict prior knowledge of encryption schemes. In this paper, we introduce a new end-to-end method of analyzing the visual security of perceptually encrypted images, without any manual work or knowing any prior knowledge of the encryption scheme. Specifically, by leveraging convolutional neural networks (CNNs), we propose a progressive recovery network (PRNet) to recover visual content from perceptually encrypted images. Our PRNet is stacked with several dense attention recovery blocks (DARBs), where each DARB contains two branches: feature extraction branch and image recovery branch. These two branches cooperate to rehabilitate more detailed visual information and generate efficient feature representation via densely connected structure and dual-saliency mechanism. We conduct extensive experiments to demonstrate that PRNet works on different perceptual encryption schemes with different settings, and the results show that PRNet significantly outperforms the state-of-the-art CNN-based image restoration methods.

The visual security index (VSI) is a quantized indicator for objective visual security evaluation of selectively encrypted images. One challenging problem in current research is that the performance of VSIs is highly sensitive to the extracted features and the method of similarity measurement, and it is hard to choose appropriate handcrafted features from encrypted images, as well as to find an effective similarity measurement. In this paper, we make the first attempt to present a novel convolutional neural network-based visual security index (CNNVSI). Our proposed CNNVSI is purely data-driven and trained end-to-end. We propose three specialized designs to make the approach work for encrypted low-quality images without any handcrafted features or prior knowledge about the human vision system (HVS). First, we present a patch labeling algorithm to assign each encrypted patch a visual security score. Second, we design a multiscale attention residual network (MARNet) for feature learning. Last, we propose to fuse the learned features from plain images, encrypted images and their discrepancy images. Extensive and systematic experiments are conducted on five publicly available image databases to analyze the performance of our proposed CNNVSI, and the experimental results and their analysis demonstrate that our proposed CNNVSI significantly outperforms the existing state-of-the-art methods in terms of accuracy and stability.

Perceptual/selective encryption has been gaining widespread attention as an emerging technology for image privacy protection. However, few studies focus on the visual security evaluation of perceptually encrypted images, which has a significant impact on measuring the effectiveness and practicality of these encryption methods. In this paper, we propose an image importance-based visual security index (IIBVSI) by leveraging spatial contrast and texture features. Based on the characteristics of perceptually encrypted images, we present an averaged high-order gradient magnitude map to describe the spatial contrast feature and introduce a combined local amplitude map of multiple log-Gabor filters to represent the texture feature. Specifically, the multiresolution representation of an image is first created by downsampling to simulate the hierarchical property of the human visual system. Next, for each scale of image resolution, the spatial contrast and the texture feature maps are extracted from both plain and encrypted images. Similarity measurements are then conducted on these feature maps to generate the contrast and the texture similarity maps. An image importance-based pooling strategy is subsequently proposed to combine these measurements and generate a visual security score. The final IIBVSI score is computed by averaging the visual security scores of all scales of image resolution. Extensive experiments are conducted on several publicly available databases, and the results demonstrate the superiority and robustness of our proposed IIBVSI compared with existing state-of-the-art work in the low and moderate image quality ranges.

Perceptual image encryption provides an efficient and effective way to preserve the confidentiality of visual information, and the measurement of content leakage is of fundamental importance for perceptually encrypted images. Numerous visual security indexes (VSIs) have been proposed to evaluate visual content leakage. Due to the lack of perceptually encrypted image databases, image quality assessment (IQA) databases are widely adopted to evaluate the performance of existing VSIs. However, there are huge differences between VSIs and IQAs. The misuse of databases may lead to an inaccurate evaluation. In this paper, we propose a perceptually encrypted image database (PEID) which contains 1080 encrypted images from 20 plain images with 10 well-known perceptual encryption techniques. Both visual quality and content leakage scores of the encrypted images are obtained through a comprehensive subjective evaluation. We also propose a systemic methodology to accurately evaluate the monotonicity, fitness, and accuracy of VSIs. We conduct extensive experiments on the proposed PEID to evaluate the performance of existing state-of-the-art VSIs. We have made the database publicly available for download and hope that the proposed PEID can facilitate the research of visual security evaluation and beyond.

Blind image quality assessment (BIQA) aims to develop quantitative measures to automatically and accurately estimate the visual quality of an image without any prior information about its reference image. This issue has been attracting a great deal of attention for a long time; however, little work has been done on night-time images, which are crucially important for consumer photography and practical applications such as automated driving systems. In this paper, to the best of our knowledge, we conduct the first exploration on subjective and objective quality assessment of night-time images. First, we build a large-scale natural night-time image database (NNID) containing 2240 images with 448 different image contents captured by different photographic equipment in real-world scenarios. Subsequently, we carry out a subjective experiment to evaluate the perceptual quality of all the images in the NNID database. Thereafter, we perform objective assessment of night-time images by proposing a blind night-time image quality assessment metric using brightness and texture features (BNBT). Finally, extensive experiments are conducted to evaluate the performance and efficiency of the proposed BNBT metric on the NNID database. The experimental results demonstrate that this metric outperforms existing state-of-the-art BIQA methods in terms of all evaluation criteria and has an acceptable computational cost at the same time. We have made the NNID database publicly available for downloading at https://sites.google.com/site/xiangtaooo/.