Point clouds play an essential role in 3D visual media applications. Point Cloud Quality Assessment (PCQA) is vital to improving the subjective experience, but lacking an effective geometric characterization hinders its consistency with subjective perception. To tackle this problem, this article introduces a No-Reference Multi-Modal Point Cloud Quality Assessment (NR-PCQA) approach driven by adaptive geometric attention. Specifically, this article defines two dimensionless geometric descriptors, namely Radial Depth Ratio (RDR) and Relative Radial Distance (RRD). These two descriptors are then used to construct an Adaptive Geometric Attention Mechanism (AGAM), which dynamically guides the network to focus on features related to geometric quality during feature extraction. Based on AGAM, a multi-modal fusion framework is further built, where the 3D geometric cues are combined with the texture and semantic information of the 2D projection images, leading to accurate PCQA. Creatively, the Hierarchical Multi-Modal Attention Fusion (HMAF) mechanism is designed to achieve the complementary strengths of 3D and 2D features, which first maximizes the extraction of single-modal features and then deeply merges cross-modal information. Naturally, the experimental results on SJTU-PCQA and WPC databases demonstrate that the innovative design of AGAM and HMAF achieves effective multi-modal feature fusion on the basis of satisfactory description for geometric properties, resulting in higher subjective consistency than the state-of-the-art methods. Meanwhile, the proposed method exhibits strong robustness across various types of point cloud distortions and diverse point cloud content, providing comprehensive validation of its effectiveness and practicality.

With the flourishing development of multi-client interactive systems, a new type of digital image known as Screen Content Image (SCI) has emerged. Unlike traditional natural scene images, SCI encompasses various visual contents, including natural images, graphics, and text. Because the multi-region distribution characteristics of screen content images result in the presence of blank regions, malicious modifications are easier to operate and harder to perceive, making a serious threat to visual content security. To this end, this paper proposes a color screen content image hashing algorithm using adaptive text regions features and global shape features. Specifically, the text regions are adaptively collected by calculating the local standard deviation of sub-blocks. Then, quaternion Fourier significant maps are computed for the text regions, and texture statistical features are further extracted to reflect the essential visual content robustness. Moreover, the global shape features are represented from the entire color SCI to ensure the discrimination. Finally, the hash sequence with a length of 142 bits is derived from the above features. Importantly, a specialized tampering dataset for SCIs has been established, and the proposed hashing shows highly sensitive to malicious modifications with a satisfactory detection accuracy. Meanwhile, the ROC curve analysis indicates that the proposed method outperforms existing hashing algorithms.

Dense retrievers have achieved state-of-the-art performance in various information retrieval tasks, but to what extent can they be safely deployed in real-world applications?In this work, we propose a novel attack for dense retrieval systems in which a malicious user generates a small number of adversarial passages by perturbing discrete tokens to maximize similarity with a provided set of training queries.When these adversarial passages are inserted into a large retrieval corpus, we show that this attack is highly effective in fooling these systems to retrieve them for queries that were not seen by the attacker.More surprisingly, these adversarial passages can directly generalize to out-ofdomain queries and corpora with a high success attack rate-for instance, we find that 50 generated passages optimized on Natural Questions can mislead >94% of questions posed in financial documents or online forums.We also benchmark and compare a range of state-ofthe-art dense retrievers, both unsupervised and supervised.Although different systems exhibit varying levels of vulnerability, we show they can all be successfully attacked by injecting up to 500 passages, a small fraction compared to a retrieval corpus of millions of passages.1

Perceptual image hashing is an effective and efficient way to identify images in large-scale databases, where two major performances are robustness and discrimination. A better tradeoff between robustness and discrimination is still a severe challenge for the current hashing research. Aiming at this issue, we design a novel perceptual image Hashing with Locality Preserving Projection (LPP) (hereafter HLPP). Specifically, to improve the robustness against content-preserving operations, Gabor filtering is leveraged to adaptively extract the orientation and structure features, which are consistent with the response of human visual system. The LPP is adopted to learn intrinsic local structure from the maximum Gabor filtering response. The use of LPP can discover meaningful low-dimensional information hidden in the maximum Gabor filtering response and thus improves discrimination of HLPP. During hash similarity calculation, the Hamming distance is selected as the metric. The tradeoff performance between robustness and discrimination is validated on benchmark databases, and the results indicate that the proposed HLPP is superior to some state-of-the-art algorithms. In addition, extensive experiments of copy detection also demonstrate that the proposed HLPP can provide higher accuracy than the compared algorithms.

Image copy detection is an important technology of copyright protection. This paper proposes an efficient hashing method for image copy detection using 2D-2D (two-directional two-dimensional) PCA (Principal Component Analysis). The key is the discovery of the translation invariance of 2D-2D PCA. With the property of translation invariance, a novel model of extracting rotation-invariant low-dimensional features is designed by combining PCT (Polar Coordinate Transformation) and 2D-2D PCA. The PCT can convert an input rotated image to a translation matrix. Since the 2D-2D PCA is invariant to translation, the low-dimensional features learned from the translation matrix are rotation-invariant. Moreover, vector distances of low-dimensional features are stable to common digital operations and thus hash construction with the vector distances is of robustness and compactness. Three open image datasets are exploited to conduct various experiments for validating efficiencies of the proposed method. The results demonstrate that the proposed method is much better than some representative hashing methods in the performances of classification and copy detection.

Nowadays the problem of image quality assessment (IQA) for screen content images (SCIs) has become a research hotspot as they are ubiquitous in multimedia applications. Although the quality assessment of natural images (NIs) has been continuously developed in the past few decades, few NI-oriented IQA methods can be directly applied on SCIs due to different visual characteristics between them. In this paper, we present a no-reference quality prediction approach considering the content information of SCIs, which is based on dual-channel multi-task convolutional neural network. First, we segment a SCI into small patches and classify them as the textual patches and the pictorial patches. Then, we devise a novel dual-channel convolutional neural network (CNN) to predict the quality of textual patches and pictorial patches. Finally, we propose an effective adaptive weighting strategy for quality score aggregation. The proposed CNN is built on an end-to-end multi-task learning framework, which assists the SCI quality prediction task through the histogram of oriented gradient (HOG) feature prediction task to learn a better mapping between the input patch and its quality score. The adaptive weighting strategy further improves the representation ability of each SCI patch. Experimental results on two largest SCI-oriented databases demonstrate that the proposed method outperforms most of the state-of-the-art no-reference IQA methods and the full-reference IQA methods.

Numerous screen content images (SCIs) have been produced to meet the needs of virtual desktop and remote display, which put forward a very urgent requirement for security and management of SCIs. Perceptual hashing is an effective way to deal with this issue. However, since SCIs are generally composed of pictures, graphics and texts, their intrinsic characteristics are different from those of natural images. Thus the previous hashing methods for natural images are not suitable for SCIs. In this article, we propose a perceptual hashing method for SCIs from the perspective of visual content understanding. Specifically, considering that the visual content understanding of SCIs mainly comes from textual regions, while the contours of text always have thinner width and higher contrast, it is decided to generate hash in the gradient field. An input screen image is first performed by some joint preprocessing operations. Then the maximum gradient magnitude and corresponding orientation information are extracted from three color channels R, G and B. Normalized histogram and local frequency coefficient features are further obtained from the maximum gradient magnitude. Finally, a hash sequence is constructed by statistics that are derived from extracted features. Experiments validated on three SCIs databases were conducted to evaluate classification between robustness and discrimination. Receiver operating characteristics (ROC) results demonstrate that the proposed method is superior to the state-of-the-art algorithms. Besides, SIQAD and SCID databases were leveraged to present the application in reduced-reference screen content image quality assessment, and comparisons show that our hashing could provide accurate predictions than other metrics.

Content-based image copy detection has become one of the important technologies in copyright protection, where two major processes, content-based feature extraction and matching are included. However, it is certainly true that enough storage space is required to establish feature database for matching, which greatly increases time and storage consumption, as well as lacks flexibility. Fortunately, perceptual image hashing is a good strategy to address these problems, in which content-based features are extracted and further encoded to hash codes. On the one hand, content-based features provide and ensure higher copy detection accuracy, while on the other hand, hash codes instead of feature database reduce storage space and improve time efficiency. Meanwhile, a better balance between robustness and discrimination is one of the most objectives of image hashing, which is conducive to its application in multimedia management and security. Consequently, we present an effective image hashing method for copy detection. Specifically, to obtain perceptual robustness against to copy attacks, we extract the global statistical characteristics in gray-level co-occurrence matrix (GLCM) to reveal texture changes. Then, to make up the discrimination limitation, we leverage the local dominant DCT coefficients from the first row/column in each sub-image to calculate vector distance. Finally, two kinds of complementary information (global feature via texture and local feature via vector distance) are simultaneously preserved to generate hash codes. Various experiments performed on benchmark database indicate that our proposed perceptual image hashing provides higher detection accuracy and better balance between robustness and discrimination than the state-of-the-art algorithms.

Hashing method is an efficient technique of multimedia security for content protection. It maps an image into a content-based compact code for denoting the image itself. While most existing algorithms focus on improving the classification between robustness and discrimination, little attention has been paid to geometric invariance under normal digital operations, and therefore results in quite fragile to geometric distortion when applied in image copy detection. In this article, a novel effective image hashing method is proposed based on geometric invariant vector distance in both spatial domain and frequency domain. First, the image is preprocessed by some joint operations to extract robust features. Then, the preprocessed image is randomly divided into several overlapping blocks under a secret key, and two different feature matrices are separately obtained in the spatial domain and frequency domain through invariant moment and low frequency discrete cosine transform coefficients. Furthermore, the invariant distances between vectors in feature matrices are calculated and quantified to form a compact hash code. We conduct various experiments to demonstrate that the proposed hashing not only reaches good classification between robustness and discrimination, but also resists most geometric distortion in image copy detection. In addition, both receiver operating characteristics curve comparisons and mean average precision in copy detection clearly illustrate that the proposed hashing method outperforms state-of-the-art algorithms.

Image hashing is a novel technology of multimedia processing with wide applications. Robustness and discrimination are two of the most important objectives of image hashing. Different from existing hashing methods without a good balance with respect to robustness and discrimination, which largely restrict the application in image retrieval and copy detection, i.e., seriously reducing the retrieval accuracy of similar images, we propose a new hashing method which can preserve two kinds of complementary features (global feature via texture and local feature via DCT coefficients) to achieve a good balance between robustness and discrimination. Specifically, the statistical characteristics in gray-level co-occurrence matrix (GLCM) are extracted to well reveal the texture changes of an image, which is of great benefit to improve the perceptual robustness. Then, the normalized image is divided into image blocks, and the dominant DCT coefficients in the first row/column are selected to form a feature matrix. The Euclidean distance between vectors of the feature matrix is invariant to commonly-used digital operations, which helps make hash more compact. Various experiments show that our approach achieves a better balance between robustness and discrimination than the state-of-the-art algorithms.

We propose a novel perceptual image hashing based on weighted discrete wavelet transform (DWT) statistical features. This hashing converts input image into a normalized image by bi-linear interpolation and color space conversion, extracts edge image of the normalized image via Canny operator, and divides the edge image into non-overlapping blocks. For each block, a three-level 2D DWT is applied to obtain different sub-bands and the weighted sum of the DWT statistics of these sub-bands is calculated. Finally, image hash is generated by concatenating and quantizing these weighted DWT features. Similarity of image hashes is measured by Euclidean distance. The Copydays dataset and the Uncompressed Color Image Database (UCID) are both used to evaluate classification between robustness and discrimination. Receiver operating characteristics curve comparisons illustrate that our hashing is superior to some state-of-the-art algorithms in classification performance with respect to robustness and discrimination. The LIVE Image Quality Assessment Database is used to validate our application in reduced-reference image quality assessment. Experimental results show that our hashing has better performance in image quality assessment than two popular measures, i.e. peak signal-to-noise ratio and structural similarity.