UNsupervised Image-to-image Translation (UNIT) aims to translate images across visual domains without paired training data, which has been widely used in style transfer, image processing, game design, etc. However, ensuring the correspondence (e.g., target category, pose, or head orientation) between generated images and inputs remains a formidable challenge. To this end, we present a new scheme, named EC-UNIT, which comprises three innovative designs aiming to Enhance cross domain Correspondence for UNIT. Specifically, 1) we propose Multi-level Style Embedding to extract multi-level style features for fusion while imposing our newly designed Hierarchical Consistency Constraints on both the content and style features (MSE&HCC), aiming to retain more style representations and facilitate feature disentanglement; 2) we develop Semantic Perceptual Matching (SPM) to minimize the semantic distribution discrepancy between the generated image and the input image by leveraging the multimodal model CLIP, dedicated to enhancing semantic consistency; 3) considering that previous works have struggled to control the image translation using pixel-level visual consistency constraints, we design Visual Perceptual Guidance (VPG) to reduce the perceptual distance between the generated image and the style input in VGG feature space, devoted to enhancing visual perceptual correspondence, thereby preventing the generation of unrealistic image details. Extensive experiments demonstrate that our EC-UNIT is more stable and outperforms current SOTA competitors in terms of image quality and diversity as well as both content and style consistency.

The defect on the fabric surface is one of the important factors affecting the quality of fabrics. Defect detection becomes the core means of quality control. Current deep-Iearning-based defect detection methods present a significant challenge in detection accuracy due to the diversity of fabric patterns and the scarcity of defect samples. Inspired by the transfer-learning paradigm, this article proposes a novel YOLOv7-based fine-tuning method for fabric defect detection. Specifically, we first employ a well-trained object detection network (i.e. YOLOv7) as a benchmark model to locate and classify defect-picking points. Considering the fabrics are mostly occluded and shaded, we therefore introduce a multi-channel attention mechanism and design the corresponding loss function. Finally, we manually clean a fabric defect dataset and use it to fine-tune the YOLOv7. Extensive experiments tested on public datasets show that our fine-tuned YOLOv7 improves mAP by 3.5%, precision by 6.3%, and recall rate by 5.7% compared to the baseline model YOLOv7. Especially, our fine-tuning method leads to a 6 MB reduction in model size. Further ablation studies demonstrate the effectiveness and contribution of our designs toward the whole network model.

Contrastive quantization (applying vector quantization to contrastive learning) has achieved great success in large-scale image retrieval because of its advantage of high computational efficiency and small storage space. This article designs a novel optimization framework to simultaneously optimize the cross quantization and the debiased contrastive learning, termed Debiased Cross Contrastive Quantization (DCCQ). The proposed framework is implemented in an end-to-end network, resulting in both reduced quantization error and deletion of many false negative samples. Specifically, to increase the distinguishability between codewords, DCCQ introduces the codeword similarity loss and soft quantization entropy loss for network training. Furthermore, the memory bank strategy and multi-crop image augmentation strategy are employed to promote the effectiveness and efficiency of contrastive learning. Extensive experiments on three large-scale real image benchmark datasets show that the proposed DCCQ yields state-of-the-art results.

Self-similarity techniques are booming in blind super-resolution (SR) due to accurate estimation of the degradation types involved in low-resolution images. However, high-dimensional matrix multiplication within self-similarity computation prohibitively consumes massive computational costs. We find that the high-dimensional attention map is derived from the matrix multiplication between query and key, followed by a softmax function. This softmax makes the matrix multiplication inseparable, posing a great challenge in simplifying computational complexity. To address this issue, we first propose a second-order Taylor expansion approximation (STEA) to separate the matrix multiplication of query and key, resulting in the complexity reduction from$\mathcal {O}(N^{2})$to$\mathcal {O}(N)$. Then, we design a multi-scale large field reception (MLFR) to compensate for the performance degradation caused by STEA. Finally, we apply these two core designs to laboratory and real-world scenarios by constructing LabNet and RealNet, respectively. Extensive experimental results tested on five synthetic datasets demonstrate that our LabNet sets a new benchmark in qualitative and quantitative evaluations. Tested on the real-world dataset, our RealNet achieves superior visual quality over existing methods. Ablation studies further verify the contributions of STEA and MLFR towards both LabNet and RealNet frameworks. Codes are available athttps://github.com/GZHU-DVL/STEA-MLFR.

Universal adversarial patch attacks pose a significant threat to deep models since a single patch can be applied to massive images yielding misclassification. One pioneering work called HARDBEAT [1] has been developed by combining gradient estimation and genetic algorithm (GA) to generate the universal adversarial patch. However, HARDBEAT can produce only a limited number of patch patterns to optimize the adversarial patch, resulting in the premature convergence of GA without achieving the universal patches with a high attack success rate (ASR). In this article, we propose an improved HARDBEAT (ImHARDBEAT) wherein an optimized GA is presented to overcome such a premature convergence issue. Specifically, our ImHARDBEAT designs a conditional crossover operation that can retain the higher ASR patterns during later iterations. Furthermore, we introduce a large mutation rate to expand the exploring space, dramatically reducing the probability of local optima. Extensive experiments are conducted on four popular datasets, involving eight models. Experimental results demonstrate the superiority of our ImHARDBEAT over current state-of-the-art methods including HARDBEAT.

Deep neural networks are fragile to attacks from adversarial examples. However, successfully fooling a target model with a limited query budget is challenging in black-box scenarios where none of the network architecture, parameters, or training data is available. An alternative solution is to employ a generative adversarial network (GAN) to generate synthetic data and train a substitute model of the target model, allowing us to perform the white-box attack, namely the transfer attack. We find that the current single network substitution suffers from a performance bottleneck. This article presents a multi-network ensembling to optimize GAN training and adversarial example generation, which includes a new multi-network substitute training strategy and an adaptive ensemble attack strategy. Extensive experiments on the MNIST and CIFAR-10 datasets show that our method outperforms the state-of-the-art in terms of query efficiency. Especially, when attacking the Microsoft Azure online model in both the label-only and probability-only scenarios, our method achieves a 100% attack success rate with a meager query budget. The code is available at https://github.com/GZHU-DVL/ZHENG.

Recent studies have shown that deep learning-based classifiers are vulnerable to malicious inputs, i.e., adversarial examples. A practical solution is to construct a perceptible but localized perturbation called patch, making the well-trained models misclassified. However, most existing patch-based adversarial attacks focus on designing patches with localized rectangles, squares, or grids, ignoring the effect of the non-local patch. In this paper, we propose a novel cross-shaped patch attack paradigm (CSPA), a simple yet efficient and effective adversarial attack in Black-box scenarios. Specifically, the cross-shaped patch consists of two line segments intersected and perpendicular to each other at the midpoint. These two line segments are designed to be sufficiently thin and long to reach the four corners of the input image nearly. Thus, the patch has a globalized perturbation capacity while preserving its continuousness. The content and location of cross-shaped patch are then iteratively optimized by a carefully contrived random search-based algorithm to maximize this global property. Comprehensive experiments are conducted on four benchmark datasets against various victim networks. The results show that the proposed CSPA outperforms the existing patch-based attacks regarding both attack success rate and query efficiency by a large margin. Specifically, compared with the baselines, CSPA increases the success rate by up to 20% on ImageNet and reaches 100% on the CIFAR-100 and CIFAR-10 datasets. Meanwhile, CSPA reduces the average number of queries by up to 7 times. Even for the white-box attack scenario, our designed cross-shaped patch can still be applicable, achieving state-of-the-art performance.

The quality of videos is the primary concern of video service providers. Built upon deep neural networks, video quality assessment (VQA) has rapidly progressed. Although existing works have introduced the knowledge of the human visual system (HVS) into VQA, there are still some limitations that hinder the full exploitation of HVS, including incomplete modeling with few HVS characteristics and insufficient connection among these characteristics. In this article, we present a novel spatial-temporal VQA method termed HVS-5M, wherein we design five modules to simulate five characteristics of HVS and create a bioinspired connection among these modules in a cooperative manner. Specifically, on the side of the spatial domain, the visual saliency module first extracts a saliency map. Then, the content-dependency and the edge masking modules extract the content and edge features, respectively, which are both weighted by the saliency map to highlight those regions that human beings may be interested in. On the other side of the temporal domain, the motion perception module extracts the dynamic temporal features. Besides, the temporal hysteresis module simulates the memory mechanism of human beings and comprehensively evaluates the video quality according to the fusion features from the spatial and temporal domains. Extensive experiments show that our HVS-5M outperforms the state-of-the-art VQA methods. Ablation studies are further conducted to verify the effectiveness of each module toward the proposed method. The source code is available at https://github.com/GZHU-DVL/HVS-5M.

Deep recognition models aim to recognize targets with various quality levels in uncontrolled application circumstances, and typically low-quality images usually retard the recognition performance dramatically. As such, a straightforward solution is to restore low-quality input images as pre-processing during deployment. However, this scheme cannot guarantee that deep recognition features of the processed images are conducive to recognition accuracy. How deep recognition features of low-quality images can be refined during training to optimize recognition models has largely escaped research attention in the field of metric learning. In this paper, we propose a quality-aware feature refinement framework based on the dedicated quality priors obtained according to the recognition performance, and a novel quality self-distillation algorithm to learn recognition models. We further show that the proposed scheme can significantly boost the performance of the recognition model with two popular deep recognition tasks, including face recognition and person re-identification. Extensive experimental results provide sufficient evidence on the effectiveness and impressive generalization capability of the proposed framework. Moreover, our framework can be essentially integrated with existing state-of-the-art classification loss functions and network architectures, without extra computation costs during deployment. The source code is available athttps://github.com/oufuzhao/QSD

Recent studies have shown that image structure can significantly facilitate image inpainting. However, current approaches mostly explore structure prior without considering its guidance to texture reconstruction, leading to performance degradation. To solve this issue, we propose a two-stream Tracking Structure Transformer (TSFormer), including structure target stream and image completion stream, to capture the synchronous and dynamic interplay between structure and texture. Specifically, we first design a structure enhancement module to restore the Histograms of Oriented Gradient (HOG) and the edge of an input image in a sketch space, which forms the input of the structure target stream. Meanwhile, in the image completion stream, we design a channel-space parallel-attention component to facilitate the efficient co-learning of channel and spatial visual cues. To build a bridge between the two streams, we further develop a structure-texture cross-attention module, wherein both structure and texture are synchronously extracted through self-attention, and texture extraction is implemented by dynamically tracking the structure in a cross-attention fashion, enabling the capture of the intricate interaction between structure and texture. Extensive experiments evaluated on three benchmark datasets, including CelebA, Places2, and Paris StreetView, demonstrate that the proposed TSFormer achieves state-of-the-art performance compared to its competitors. The code is available at https://github.com/GZHU-DVL/TSFormer .

In recent years, leveraging Convolutional Neural Network (CNN) to optimize Transformer (called hybrid model) has achieved great progress in image inpainting. However, the slow growth of the effective receptive field of CNN in processing large-hole regions significantly limits the overall performance. To alleviate this problem, this paper proposes a new Transformer-CNN-based hybrid framework (termed PUT+) by introducing the fast Fourier convolution (FFC) into the CNN-based refinement network. The proposed framework introduces an improved Patch-based Vector Quantized Variational Auto-Encoder (P-VQVAE+). The encoder transforms the masked region into non-overlapping patch-based unquantized feature vectors as the input of Un-Quantized Transformer (UQ-Transformer). The decoder restores the masked region from the predicted quantized features output by the UQ-Transformer while maintaining the unmasked region unchanged. Many experimental results show that the proposed method outperforms the state-of-the-art by a large margin, especially for image inpainting with large masked areas. The code is available at https://github.com/GZHU-DVL/PUTplus.

Unsupervised image-to-image translation has gained great attention in data augmentation by allowing the translation of images from one domain to another while preserving their content and style. However, existing methods face major challenges when these two domains have substantial discrepancies in shape and appearance. To overcome these challenges, we introduce a novel framework that can boost the naturalness and diversity of unsupervised image-to-image translation with pre-trained fine-grained classification (PFC-UNIT). Specifically, PFC-UNIT trains a content encoder to obtain the coarse-level content feature in the first stage. In the second stage, a new pre-trained fine-grained classification (PFC) is designed to generate fine-level images with style consistency. Furthermore, during the latter part of the second stage, a dynamic skip connection is added to generate finer-level images with content consistency. Experimental results show that as a plug-and-play tool, our PFC dramatically enhances the image translation effect by maintaining vivid details and keeping content and style consistent. And the proposed PFC-UNIT outperforms leading state-of-the-art methods. The code is available at https://github.com/GZHU-DVL/PFC-UNIT.

Image inpainting has made great progress with the help of deep learning. However, existing methods show performance degradation when restoring corrupted images with complex scenes. In this paper, we propose a novel image inpainting method by reducing intermediate layer information loss and fusing texture-structure features. To be specific, we first compute a Local Binary Pattern (LBP) map of the corrupted image as the input of structure feature extraction, considering that LBP contains richer structure information than edges and contours. Then, we introduce a Wide Identical Residual Weighting (WIRW) module to utilize the intermediate layer features in the structure encoder. Furthermore, we introduce a Spatial-Transformer (ST) module consisting of Convolutional Neural Network (CNN) and Transformer branches to fuse the structure and texture features, where the CNN and Transformer branches are responsible for capturing the local and global information, respectively. Various experiments on public datasets including CelebA, Paris StreetView, and Places2 demonstrate the effectiveness of the proposed method. Especially, our ablation study separately verifies the contribution of each module to the whole framework. The code is publicly available at https://github.com/GZHU-DVL/LFang.

The goal of unsupervised image-to-image translation (UNIT) is to translate images between two different domains without any paired data. Recent research has shown great progress in the UNIT subject, but it still faces challenges when translating between artificial and natural objects. To overcome this challenge, we propose a novel unsupervised image-to-image translation with content reconstruction (CR-UNIT), a two-stage and from-coarse-to-fine training framework. Specifically, CR-UNIT builds a corresponding relationship among the content features of different domains on a coarse granularity in the first stage. In the second stage, a new content reconstruction module is constructed to extract the fine-grained content and style features, obtaining more detailed semantic correspondence and better fusion of the content and style features. Furthermore, we design a content reconstruction loss to facilitate the training of our model. Extensive experimental results demonstrate the superiority of the proposed CR-UNIT over the existing methods. Especially for the translation task between the artificial and natural objects, our CR-UNIT achieves outstanding effect in terms of perceptive quality and objective metric. The code is available at https://github.com/GZHU-DVL/CR-UNIT.

Pursuing the precise details of super-resolution images is challenging for single-image super-resolution tasks. This paper presents a single-image super-resolution network with hairline details (termed NLCUnet), including three core designs. Specifically, a non-local attention mechanism is first introduced to restore local pieces by learning from the whole image region. Then, we find that the blur kernel trained by the existing work is unnecessary. Based on this finding, we create a new network architecture by integrating depth-wise convolution with channel attention without the blur kernel estimation, resulting in a performance improvement instead. Finally, to make the cropped region contain as much semantic information as possible, we propose a random 64×64 crop inside the central 512×512 crop instead of a direct random crop inside the whole image of 2K size. Numerous experiments conducted on the benchmark DF2K dataset demonstrate that our NLCUnet performs better than the state-of-the-art in terms of the PSNR and SSIM metrics and yields visually favorable hairline details.

No-Reference Video Quality Assessment (NR-VQA) plays an essential role in improving the viewing experience of end-users. Driven by deep learning, recent NR-VQA models based on Convolutional Neural Networks (CNNs) and Transformers have achieved outstanding performance. To build a reliable and practical assessment system, it is of great necessity to evaluate their robustness. However, such issue has received little attention in the academic community. In this paper, we make the first attempt to evaluate the robustness of NR-VQA models against adversarial attacks, and propose a patch-based random search method for black-box attack. Specifically, considering both the attack effect on quality score and the visual quality of adversarial video, the attack problem is formulated as misleading the estimated quality score under the constraint of just-noticeable difference (JND). Built upon such formulation, a novel loss function called Score-Reversed Boundary Loss is designed to push the adversarial video’s estimated quality score far away from its ground-truth score towards a specific boundary, and the JND constraint is modeled as a strict $L_2$ and $L_\infty$ norm restriction. By this means, both white-box and black-box attacks can be launched in an effective and imperceptible manner. The source code is available at https://github.com/GZHU-DVL/AttackVQA.

Approximate nearest neighbour search (ANNS) in high-dimensional space is an essential and fundamental operation in many applications from many domains such as multimedia database, information retrieval and computer vision. With the rapidly growing volume of data and the dramatically increasing demands of users, traditional heuristic-based ANNS solutions have been facing great challenges in terms of both efficiency and accuracy. Inspired by the recent successes of deep learning in many fields, substantial efforts have been devoted to applying deep learning techniques to ANNS for learning to index and learning to search, resulting in numerous algorithms that achieve state-of-the-art performance compared with conventional methods. In this survey paper, we comprehensively review the different types of deep learning-based ANNS methods according to two learning paradigms:learning to indexandlearning to search. We provide a comprehensive overview and analysis of these methods in a systematic manner. Based on the overview, we point out thatend-to-end learningwill be a new and promising research direction for deep learning-based ANNS, i.e., applying deep learning techniques to jointly learn the indexing and searching together, such that the underlying knowledge learned from data can directly contribute to the final searching performance. Finally, we conduct experiments and provide general performance analyses for the representative deep learning-based ANNS algorithms.

The security of spread spectrum (SS) watermarking largely depends on the difficulty of estimating its secret key. Some estimators have been proposed to estimate the secret key in the known-message attack (KMA) scenario. However, the estimation accuracies of existing estimators are not satisfactory when the number of observations is not large enough. Currently, it is still a challenging and open problem to design more effective estimators. In this paper, we propose an equivalent keys (EK)-based estimator to estimate the secret key for both the traditional and more secure SS watermarking methods. Equivalent keys form an equivalent region, which is the intersection of a unit hypersphere and a hypercone. According to the Monte Carlo simulation, we find that the secret key can be estimated by adding up the equivalent keys uniformly sampled from the equivalent region. Thus, the proposed estimator selects equivalent keys from randomly-generated vectors by exploiting the pairs of watermarked signals and their embedded messages. A theoretical analysis is performed for the proposed estimator to evaluate the estimation accuracy. Experimental results verify the theoretical analysis and show the superiority of the proposed estimator over existing estimation methods. Furthermore, this paper also shows the insecurity of the more secure SS watermarking methods in the KMA scenario from a practical perspective for the first time.

Automatically evaluating the quality of in-the-wild videos is challenging since both the distortion types and reference videos are unknown. In general, humans can make a fast and accurate judgment for video quality. Fortunately, deep neural networks have been developed to effectively model the human visual system (HVS). In this paper, we deeply investigate three elements of HVS, including texture masking, content-dependency, and temporal-memory effects from an experimental perspective. Based on the investigation, we propose to make full use of texture information to boost the performance of video quality assessment (VQA), termed TiVQA in this paper. To be specific, TiVQA first uses the local binary pattern (LBP) operator to detect texture information of each video frame. Then a two-stream ResNet is employed to extract the texture masking and content-dependency embeddings, respectively. Finally, TiVQA integrates both the gated recurrent unit and subjectively-inspired temporal pooling layer to model the temporal-memory effects. Extensive experiments on benchmark datasets including KoNViD-1k, CVD2014, LIVE-Qualcomm, and LSVQ show that the proposed TiVQA obtains state-of-the-art performance in terms of SRCC and PLCC.

Domain gap between synthetic and real rain has impeded advances in natural image deraining task. Existing methods are mostly built on convolutional neural networks (CNNs) and the receptive field of CNNs is limited, thereby resulting in poor domain adaptation. This paper designs a dual-domain translation Transformer network (termed DTT-Net) for semi-supervised image deraining. By leveraging Transformer architecture, the proposed DTT-Net can significantly mitigate the domain gap, greatly boosting the performance on real-world rainy images. Meanwhile, DTT-Net integrates three loss functions including adversarial, cycle-consistency, and MSE losses to adversarial training to further improve the visual quality of the derained images. Extensive experiments are conducted on synthetic and real-world rain datasets. Experimental results show that our DTT-Net outperforms the state-of-the-art by more than 2 dB PSNR. The source code is available at https://github.com/GZHU-DVL/DTT-Net.

We study the hard-label adversarial attacks where model information, training data, and output score are all unknown except for the final decision to an input query. Due to the security issue, model providers usually constrain the number of queries. It is challenging for adversaries to attack a model with limited queries. Sign-OPT makes great advances in query complexity, which adopts the backtracking line-search to find the optimal search direction, meanwhile the binary search is employed to obtain the minimum-distortion adversarial example. We find that this binary search costs a huge amount of queries. This paper proposes an improved Sign-OPT, termed Sign-OPT+, to enhance query efficiency further. Instead of the binary search, at each line-search stage we directly judge whether the candidate example along the new search direction locates inside or outside the decision boundary. This judgment requires only one query to achieve the optimal search direction, significantly reducing the overall queries. Experiments tested on MNIST, CIFAR-10, and ImageNet show that our Sign-OPT+ requires fewer queries and obtains a higher success rate than the state-of-the-art including Sign-OPT. The source code is available at https://github.com/GZHU-DVL/Sign-OPT-plus.

In order to be applicable in real-world scenario, Boundary Attacks (BAs) were proposed and ensured one hundred percent attack success rate with only decision information. However, existing BA methods craft adversarial examples by leveraging a simple random sampling (SRS) to estimate the gradient, consuming a large number of model queries. To overcome the drawback of SRS, this paper proposes a Latin Hypercube Sampling based Boundary Attack (LHS-BA) to save query budget. Compared with SRS, LHS has better uniformity under the same limited number of random samples. Therefore, the average on these random samples is closer to the true gradient than that estimated by SRS. Various experiments are conducted on benchmark datasets including MNIST, CIFAR, and ImageNet-1K. Experimental results demonstrate the superiority of the proposed LHS-BA over the state-of-the-art BA methods in terms of query efficiency. The source codes are publicly available at https://github.com/GZHU-DVL/LHS-BA.

Transformer based on self-attention mechanism is blooming in computer vision nowadays. However, its application to video quality assessment (VQA) has not been reported. Evaluating the quality of in-the-wild videos is challenging due to the unknown of pristine reference and shooting distortion. This paper presents a novel space-time attention network for the VQA problem, named StarVQA. StarVQA builds a Transformer by alternately concatenating the divided space-time attention. To adapt the Transformer architecture for training, StarVQA designs a vectorized regression loss by encoding the mean opinion score (MOS) to the probability vector and embedding a special vectorized label token as the learnable variable. To capture the long-range spatiotemporal dependencies of a video sequence, StarVQA encodes the space-time position information of each patch to the input of the Transformer. Various experiments are conducted on the de-facto in-the-wild video datasets, including LIVE-VQC, KoNViD-1k, LSVQ, and LSVQ-1080p. Experimental results demonstrate the superiority of StarVQA over the state-of-the-art. The source code is available at https://github.com/GZHU-DVL/StarVQA.

Image inpainting has achieved great advances by simultaneously leveraging image structure and texture features. However, due to lack of effective multi-feature fusion techniques, existing image inpainting methods still show limited improvement. In this paper, we design a deep multi-feature co-learning network for image inpainting, which includes Soft-gating Dual Feature Fusion (SDFF) and Bilateral Propagation Feature Aggregation (BPFA) modules. To be specific, we first use two branches to learn structure features and texture features separately. Then the proposed SDFF module integrates structure features into texture features, and meanwhile uses texture features as an auxiliary in generating structure features. Such a co-learning strategy makes the structure and texture features more consistent. Next, the proposed BPFA module enhances the connection from local feature to overall consistency by co-learning contextual attention, channel-wise information and feature space, which can further refine the generated structures and textures. Finally, extensive experiments are performed on benchmark datasets, including CelebA, Places2, and Paris StreetView. Experimental results demonstrate the superiority of the proposed method over the state-of-the-art. The source codes are available at https://github.com/GZHU-DVL/MFCL-Inpainting.

This paper describes an ultra high definition (UHD) video dataset named DVL2021 for the perceptual study of video quality assessment (VQA). To our knowledge, DVL2021 is the first authentically distorted 4K (3840 × 2160) UHD video quality dataset. The dataset contains 206 versatile 4K UHD video sequences, which are all collected in in-the-wild scenarios. Each sequence is captured at 50 frames per second (fps), stored in raw 10-bit 4:2:0 YUV format, and has a duration of 10 s. Following the subjective evaluation method of TV image quality granted by ITU-R BT.500-13, 32 unique participants take part in the manual annotation process, whose ages are from teenage to sixties (32.7 years old on average). DVL2021 has the following merits: (1) enormous variety of video contents, (2) captured by different types of cameras, (3) complex types and multiple levels of authentic distortion, (4) broadly distributed temporal/spatial information, and (5) a wide spectrum of mean opinion scores (MOS) distribution. Furthermore, we conduct a benchmark experiment by evaluating several mainstream VQA methods on DVL2021. The baseline results are higher than 0.75 in Spearman's rank order correlation coefficient (SROCC) metric. Our study provides a basis for the UHD VQA problem. DVL2021 is publicly available at https://github.com/GZHU-DVL/DVL2021.

In black-box attack, excessive queries to target model may cause suspicion and expose attacker's identity. Equipped with advanced meta learning technique, Meta Attack simulates the target model with a surrogate model, significantly reducing the queries. However, it queries for ZOO-gradients to correct the estimated meta-gradients with a fixed frequency, thereby still leading to massive unnecessary queries. To overcome this limitation, this letter takes the dynamic changes of the accuracy of the estimated gradients as a starting point, and develops a Dynamic Meta Attack (DMA). At the beginning of each fine-tuning round, DMA computes the distance between the above two types of gradients. Such distance metric can reflect the accuracy of the meta-gradients, and guide the dynamic adjustment of query frequency for the ZOO-gradients. Moreover, the working flow of the dynamic fine-tuning process can be controlled by a set of parameters, which are of physical significance and easy to be tuned. By this means, DMA merely launches queries at critical moments, greatly saving query resource. Experiments conducted on MNIST and CIFAR10 show that the proposed DMA requires far fewer queries than existing methods while maintaining a satisfying attack success rate and distortion.

Over the past two decades, spread spectrum (SS) embedding has been widely used in digital watermarking due to its competitive performance in robustness and security. However, the robustness of existing secure SS embedding methods, such as natural watermarking (NW) and robust-NW (RNW), is still weak. In this article, we propose a new secure SS embedding method named truncated-RNW (TRNW), which improves the robustness of RNW while maintaining the same security level. The main idea of TRNW is to move RNW-watermarked correlations within an origin-centered sphere onto the spherical surface along the radial direction. Moreover, Hungarian algorithm is used to reduce embedding distortion, and the optimized method is called Hungarian-TRNW (HTRNW). A theoretical analysis and extensive experiments are conducted to validate the effectiveness of the proposed method. The results show that HTRNW achieves the same security level as RNW and a significant improvement over existing representative secure SS embeddings in terms of robustness.

Recently, applying deep learning to no-reference image quality assessment (NR-IQA) has received significant attention. Especially in the last five years, an increasing interest has been drawn to the studies of rank learning since it can help mitigate the problem of small IQA datasets. However, on one hand, existing rank learning is not suitable for the authentically distorted images due to the lack of generated rank samples. On the other hand, the output of existing rank loss functions is uncontrollable, resulting in reduced performance. Motivated by these two limitations, we propose a novel rank learning based NR-IQA method, termed controllable list-wise ranking IQA (CLRIQA) in this paper. To be specific, we first present an imaging-heuristic approach, in which the over- and under-exposure is formulated as an inverse of the Weber-Fechner law, and fusion strategy and compression are adopted, to simulate the authentic distortion and generate the rank image samples. These samples are label-free yet associated with quality ranking information. Then we design a controllable list-wise ranking (CLR) loss function by setting an upper and lower bound of rank range and introducing an adaptive margin to tune rank interval. Finally, both the generated rank samples and proposed CLR are used to pre-train a convolutional neural network. Moreover, to obtain a more accurate prediction model, we take advantage of the IQA datasets to fine-tune the pre-trained network further. Various experiments are conducted on the IQA benchmark datasets, and experimental results demonstrate the effectiveness of the proposed CLRIQA method. The source code and network model can be downloaded at the following web address:https://github.com$/$GZHU-DVL$/$CLRIQA.

In recent years, Face Image Quality Assessment (FIQA) has become an indispensable part of the face recognition system to guarantee the stability and reliability of recognition performance in an unconstrained scenario. For this purpose, the FIQA method should consider both the intrinsic property and the recognizability of the face image. Most previous works aim to estimate the sample-wise embedding uncertainty or pair-wise similarity as the quality score, which only considers the partial information from the intra-class. However, these methods ignore the valuable in-formation from the inter-class, which is for estimating the recognizability of face image. In this work, we argue that a high-quality face image should be similar to its intra-class samples and dissimilar to its inter-class samples. Thus, we propose a novel unsupervised FIQA method that incorporates Similarity Distribution Distance for Face Image Quality Assessment (SDD-FIQA). Our method generates quality pseudo-labels by calculating the Wasserstein Distance (WD) between the intra-class and inter-class similarity distributions. With these quality pseudo-labels, we are capable of training a regression network for quality prediction. Extensive experiments on benchmark datasets demonstrate that the proposed SDD-FIQA surpasses the state-of-the-arts by an impressive margin. Meanwhile, our method shows good generalization across different recognition systems.

Lattice code has been widely used for data hiding. It can provide security for data hiding by randomly translating its codebook with a secret dither. However, besides the secret dither, there are an infinite amount of points that are near the secret dither and can also be used for perfect decoding in the noiseless scenario. This means that lattice-based data hiding has a serious security hole, named decoding security hole (DSH) in this paper. After a theoretical analysis of DSH, we find that these points form a convex polytope and the centroid of this convex polytope is the secret dither. Based on this finding, a simple yet effective attack method is presented to estimate the secret dither of lattice-based data hiding. Extensive experimental results show that the proposed method significantly outperforms state-of-the-art attack methods, especially when the number of observations is small or the document-to-watermark ratio changes over a wide range.

Visual secret sharing (VSS) is a new technique for sharing a binary image into multiple shadows. For VSS, the original image can be reconstructed from the shadows in any qualified set, but cannot be reconstructed from those in any forbidden set. In most traditional VSS schemes, the shadows held by participants have the same importance. However, in practice, a certain number of shadows are given a higher importance due to the privileges of their owners. In this article, a novel ( t , s , k , n )-threshold VSS scheme is proposed based on access structure partition. First, we construct the basis matrix of the proposed ( t , s , k , n )-threshold VSS scheme by utilizing a new access structure partition method and sub-access structure merging method. Then, the secret image is shared by the basis matrix as n shadows, which are divided into s essential shadows and n - s non-essential shadows. To reconstruct the secret image, k or more shadows should be collected, which include at least t essential shadows; otherwise, no information about the secret image can be obtained. Compared with related schemes, our scheme achieves a smaller shadow size and a higher visual quality of the reconstructed image. Theoretical analysis and experiments indicate the effectiveness of the proposed scheme.

In this article, we propose a(τ, ε)-greedy reinforcement learning algorithm for anti-jamming wireless communications, which chooses previous action with probability τ and applies ε-greedy with probability 1-τ. The key idea of our algorithm is that the more valuable the previous action is, the higher probability of directly performing it at the current time slot without learning. For this purpose, the average utility of several previous actions is first calculated as a threshold for the valuable action judgment. Then, probability τ is formulated as a Gaussian-like function with respect to the difference between the threshold and the utility of the previous action, which makes the wireless devices find the optimal action at a faster speed in the early stage, and eventually ensures the convergence. As a concrete example, the proposed algorithm is implemented in a wireless communication system against multiple jammers. Simulation results show that compared with ε-greedy, the (τ, ε)-greedy obtains faster convergence rate and slightly higher signalto-interference-plus-noise ratio when being applied to Qlearning, deep Q-networks (DQN), double DQN (DDQN), and prioritized experience reply based DDQN (PDDQN). The source code is available at https://github.com/GZHUDVL/tau-epsilon-greedy-RL.

Natural scene statistics (NSS) model has received considerable attention in the image quality assessment (IQA) community due to its high sensitivity to image distortion. However, most existing NSS-based IQA methods extract features either from spatial domain or from transform domain. There is little work to simultaneously consider the features from these two domains. In this paper, a novel blind IQA method (NBIQA) based on refined NSS is proposed. The proposed NBIQA first investigates the performance of a large number of candidate features from both the spatial and transform domains. Based on the investigation, we construct a refined NSS model by selecting competitive features from existing NSS models and adding three new features. Then the refined NSS is fed into SVM tool to learn a simple regression model. Finally, the trained regression model is used to predict the scalar quality score of the image. Experimental results tested on both LIVE IQA and LIVE-C databases show that the proposed NBIQA performs better in terms of synthetic and authentic image distortion than current mainstream IQA methods. The source code is available at https://github.com/GZU-Image-Video-Lab/NBIQA.

Security analysis is a very important issue for digital watermarking. Several years ago, according to Kerckhoffs' principle, the famous four security levels, namely insecurity, key security, subspace security, and stego-security, were defined for spread-spectrum (SS) embedding schemes in the framework of watermarked-only attack. However, up to now there has been little application of the definition of these security levels to the theoretical analysis of the security of SS embedding schemes, due to the difficulty of the theoretical analysis. In this paper, based on the security definition, we present a theoretical analysis to evaluate the security levels of five typical SS embedding schemes, which are the classical SS, the improved SS (ISS), the circular extension of ISS, the nonrobust and robust natural watermarking, respectively. The theoretical analysis of these typical SS schemes are successfully performed by taking advantage of the convolution of probability distributions to derive the probabilistic models of watermarked signals. Moreover, simulations are conducted to illustrate and validate our theoretical analysis. We believe that the theoretical and practical analysis presented in this paper can bridge the gap between the definition of the four security levels and its application to the theoretical analysis of SS embedding schemes.

During the past twenty years, there has been a great interest in the study of spread spectrum (SS) watermarking. However, it is still a challenging task to design a secure and robust SS watermarking method. In this paper, we first define a family of secure SS watermarking methods, named as spherical watermarking (SW). The watermarked correlation of SW is defined to be uniformly distributed on a spherical surface, and this makes SW be key-secure against the watermarked-only attack. Then, we propose an implementation of SW, called transportation SW (TSW), which is designed to decrease embedding distortion in a recursive manner using the transportation theory, meanwhile keeping the security of SW. Moreover, we present a theoretical analysis of the embedding distortion and robustness of the proposed method. Finally, extensive experiments are conducted on simulated signals and real images. The experimental results show that TSW is more robust than existing secure SS watermarking methods.

In this paper, we first define a class of secure spread spectrum (SS) watermarking, called spherical watermarking (SW), of which the distribution of the watermarked correlations has a uniform distribution over a spherical surface. Then, we propose an implementation of SW, namely transportation SW (TSW), which recursively minimizes embedding distortion using transportation theory. A theoretical analysis is also presented on the performance of the proposed TSW scheme in terms of the watermark to content power ratio (WCR), and is validated by our simulations. Simulation results show that TSW performs better in terms of WCR and robustness than existing secure SS watermarking schemes.

Video sequence matching aims to locate a query video clip in a video database. It plays an important role in reducing storage redundancy and detecting video copies for copyright protection. In this paper, we propose an effective method for video sequence matching based on the invariance of color correlation. The proposed method first splits each key-frame into nonoverlapping blocks. For each block, we sort the red, green, and blue color components according to their average intensities, and use the percentage of the color correlation to generate a frame feature with a small size. Finally, the resulting video feature is made up of the consecutive frame features, which is demonstrated to be robust against most typical video content-preserving operations, including geometric distortion, blurring, noise contamination, contrast enhancement, and strong re-encoding. The experimental results show that the proposed method outperforms the existing methods in the literature, as well as the method based on the traditional color histogram. Furthermore, the time and space complexity of our algorithm are both satisfactory, which are very important for many real-time applications.

Although many existing steganalysis works have shown that the spatial ±1 steganography on JPEG pre-compressed images is relatively easier to be detected compared with that on the never-compressed images, most experimental results seem not very convincing since these methods usually assume that the quantization table of the JPEG stegos previously used is known before detection and/or the length of embedded message is fixed. Furthermore, there are just few effective quantitative algorithms for further estimating the spatial modifications. In this letter, we firstly propose an effective method to detect the quantization table from the contaminated digital images which are originally stored as JPEG format based on our recently developed work about JPEG compression error analysis , and then we present a quantitative method to reliably estimate the length of spatial modifications in those gray-scale JPEG stegos by using data fitting technology. The extensive experimental results show that our estimators are very effective, and the order of magnitude of prediction error can remain around measured by the mean absolute difference.

Copy detection aims to identify various content-preserving copies from the same origin and distinguish the different sources. In this paper, we propose a novel feature for image copy detection based on the radon transform (RT). We first theoretically analyze that the proposed feature is invariant against rotation, scaling and translation (RST) operations and robust to lossy JPEG compression and additive noises. The feature can meet the robustness requirements of copy detection applications. The proposed feature is statistically dependent of image content and is capable of capturing unique information of the image for fragility of the copy detection system. The experimental results show that the proposed scheme outperforms the existing methods, when detecting image copies subjected to various transformations, in terms of receiver operating characteristic (ROC) curves.

Quantization is one of the commonly used techniques in most lossy image source encoders. It is observed that the quantization operation usually introduces some obvious artifacts into the histogram of the corresponding transform coefficients under various compression schemes. By investigating such inherent artifacts over all candidate transform coefficients, it is possible to identify the transform, as well as some parameters previously employed in the transform-based encoder from a decompressed image. In this paper, we first analyze the properties of the quantized coefficients and present a simple yet effective way to detect the quantization artifacts, and then we propose an approach to identify the transform-based encoder based on the quantization artifacts detection. The simulation results evaluated on thousands of natural images with some popular compression schemes demonstrate the effectiveness of our method.