With the explosive growth of video data, learning compact discriminative binary representations for efficient video retrieval has become essential. Most existing self-supervised video hashing methods struggle to achieve consistent modeling across short-term and long-range temporal dependencies, resulting in hash codes that are less discriminative and more redundant. To address this, we propose a novel self-supervised video hashing method called CSL-Hash, based on a bidirectional Mamba encoder-decoder network. The encoder integrates residual convolution, channel attention, and bidirectional Mamba to achieve more comprehensive feature modeling, while the decoder employs bidirectional Mamba to preserve temporal semantics. Additionally, we design a new loss function consisting of reconstruction loss, contrastive loss, and orthogonality constraint loss to enhance the independence and discriminability of hash codes. Experimental results validate the effectiveness of CSL-Hash on multiple video retrieval datasets.

Video hashing is an effective method to solve copy detection problems. This paper proposes a video hashing with robust secondary frames (RSFs) and local tangent space alignment (LTSA) for copy detection. In the proposed scheme, the input video is uniformly grouped, and RSFs are calculated based on the video frame groups. Then, a tensor is obtained by stacking all RSFs, and the tensor is divided into multiple smaller tensors. The mean calculation is used to construct a matrix from these small tensors, and the low-dimensional features are extracted through LTSA. Moreover, the MobileNetV2 is used to calculate two feature maps from each RSF. The two feature maps are stacked to form a dual-channel feature map, which is then divided into multiple feature map blocks. The mean calculation is used to extract deep features from these blocks. Finally, the low-dimensional and deep features are binarized and concatenated to obtain a hash. Comparative experimental results show that our hashing scheme has the best copy detection and classification performance.

No-Reference Image Quality Assessment (NR-IQA) serves as a fundamental task in computer vision that aims to predict image quality consistent with human perception. Currently, numerous NR-IQA methods often use simplistic fusion strategies to integrate features from different backbones. However, these methods typically neglect the semantic differences and intricate inter-channel interactions among different features, thereby limiting their abilities to represent features effectively. To address this issue, we propose a Semantic-guided Channel Cross-attention Integration Network for NR-IQA (SCCIN-IQA), which enables more effective integration of complementary information from different backbones. The core module of our method is the fusion-semantic channel cross-attention. It first generates a semantic feature by integrating features from different backbones, then utilizes this semantic feature as a query to integrate the original backbone features via a channel cross-attention mechanism, thereby adaptively highlighting quality-relevant channel activations. Additionally, a space-channel enhancement module is introduced to further enhance the learned features in both space and channel dimensions, enabling comprehensive modeling of multi-dimensional contextual dependencies. Extensive experiments conducted on multiple public datasets demonstrate that the proposed SCCIN-IQA achieves state-of-the-art performance, consistently surpassing several mainstream methods while exhibiting strong generalization.

No-Reference Image Quality Assessment (NR-IQA) is an important task in the field of computer vision. Most methods utilize the pre-trained features with information irrelevant to image quality. In addition, some methods directly regress the pre-trained features without interaction or simply concatenate all features for score regression. They ignore the differences between local and global features. These issues lead to the limited IQA performance. To address this, we propose a Hybrid-Attention Refinement and Hierarchical Fusion Network (HAR-HFNet) for NR-IQA, which consists of a feature extraction module, a hybrid attention refinement module, a hierarchical dilated-attention fusion module and a quality prediction module. Firstly, the hybrid attention refinement module filters and refines the multi-stage features extracted by the pre-trained Swin Transformer, which enhances distortion-related information. Secondly, the hierarchical dilated-attention fusion module fuses the deep global feature with local features. It enables effective hierarchical integration of global semantics and local details. Finally, the quality prediction module predicts the score through weighted feature aggregation. Experiments on six public IQA datasets demonstrate that the HAR-HFNet outperforms some baseline NR-IQA methods in prediction accuracy and generalization ability.

Self-supervised video hashing aims at generating hash codes and performing fast video content retrieval by leveraging the visual content information inherent in the videos themselves. Most existing methods often overlook the structure-preserving information within the visual content of the videos and thus cannot learn an effective discriminative video representation. In this paper, a Structure-Preserving Video Hashing (SPVH) via a self-supervised Transformer for retrieval is proposed by exploring the global relationships, local relationships, and inter-video relationships in the visual content of videos. In the proposed SPVH, a Transformer-based autoencoder model is used to extract the deep features of the videos. Moreover, a new structure-preserving loss function with the clustering loss, constraint loss, contrastive loss, and reconstruction loss is designed to capture the structural information of the videos. Extensive experiments are conducted on two large-scale video datasets. The results demonstrate the superior performance of our SPVH compared to some state-of-the-art methods.

Most data-driven deep learning-based Artistic Image Aesthetics Assessment (AIAA) methods cannot effectively extract visual attributes from art images since the existing artistic image datasets don’t provide any information about visual attributes. The lack of visual attributes reduces the interpretability of AIAA methods and limits their performance. To address these problems, a novel artistic image aesthetics assessment assisted by photographic visual attributes is proposed. The proposed method consists of a feature extraction module and a joint prediction module. The feature extraction module pre-trained on a photographic dataset and an artistic image dataset can learn the information of photographic attributes and the generic artistic aesthetic information. The joint prediction module uses a non-local self-attention block to fuse the photographic visual attribute features with general artistic aesthetic features. The fused features are fed into an FC layer for calculating the artistic image aesthetic score. Experimental results indicate that our proposed method outperforms some state-of-the-art AIAA methods.

Fine-grained image retrieval (FGIR) is an important topic of image retrieval, and its challenge lies in the accurate identification of image objects with minor inter-class differences and considerable intraclass differences. Most existing methods exploit Convolutional Neural Networks (CNNs) to capture fine-grained and coarse-grained information while overlooking the scale variations. To address these issues, a novel method named Hash Generation Network (HGNet) guided by high frequency information is developed to learn crucial details across different scales. The HGNet consists of a High-Frequency Guidance Module (HFGM) and a Hash Generation Module (HGM). The key contribution is the proposed HFGM which integrates the high-frequency information and multi-scale features extracted from the Swin Transformer. As the Swin Transformer can effectively capture global contextual information, its multi-scale features, guided by high-frequency information that contains fine-grained texture details, can represent both fine-grained and coarse-grained details, thereby guiding the HGM in generating discriminative hash codes. Experimental results show that the HGNet outperforms several SOTA FGIR methods in retrieval performance.

No-Reference Image Quality Assessment (NR-IQA) is an important task of computer vision. Most deep neural networks based NR-IQA methods have the ability of accurate quality predictions, but they have large-scale parameters and high computational complexity. To alleviate these problems, we propose a lightweight NR-IQA method by unifying statistical and refined semantic features. Our proposed method consists of a lightweight feature extractor, a Statistical Semantic Feature Extraction (SSFE) module, and a Refined Semantic Feature Extraction (RSFE) module. The lightweight feature extractor is used to extract semantic features with perceptual distortion information. The SSFE module is designed to obtain statistical information of the semantic features for capturing the local and global changes of distorted image. The RSFE module is designed to refine the semantic features for measuring complex distortions. Extensive experiments on many IQA datasets are done and the results indicate that our proposed method outperforms some baseline NR-IQA methods in IQA performance, generalization ability, and model complexity.

Copy detection is crucial for protecting image copyright. This paper proposes a robust image hashing approach via Weighted Saliency Map (WSM) and Laplacian Eigenmaps (LE) (hereafter WSM-LE approach). An important contribution is the WSM construction via the edge map and the saliency map. As the WSM can indicate the interest regions of image, hash calculation based on WSM can provide robustness of our WSM-LE approach. Another contribution is the low-dimensional feature learning by the LE technique. As the LE technique can effectively learn the internal geometric relationships of image, the extracted low-dimensional features can improve discrimination of our WSM-LE approach. In addition, the low-dimensional features are treated as vectors and the vector distances are used to create a compact and encrypted hash. Numerous experiments and comparisons are conducted to confirm the effectiveness and superiority of our WSM-LE approach. The results indicate that our WSM-LE approach has excellent classification and copy detection performances than some baseline approaches.

Dividing a Screen Content Image (SCI) with complex components into pictorial and textual regions for predicting scores is one of the common Screen Content Image Quality Assessment (SCIQA) methods. However, how to efficiently leverage pictorial and textual features to predict quality scores for no-reference SCIQA still needs to be explored. In addition, statistical analysis reveals that labels of SCIs present a distribution. Therefore, both the distribution of quality scores of SCIQA and the distribution of labels need to be considered in the SCIQA. This paper proposes a no-reference SCIQA method unifying pictorial and textual features. One contribution is the proposed dual-branch extraction module with the parameter-free attention convolution block and the joint prediction module. The proposed method employs the dual-branch extraction module to generate efficient pictorial and textual features and then uses the joint prediction module to predict quality scores. Another contribution is the joint distribution loss. It makes the distribution of the quality scores as close as possible to the distribution of labels. Experiments on the SCIQA datasets show that the proposed method achieves excellent SCIQA performance and generalization ability.

Copy detection is a key task of video copyright protection. This paper presents a robust video hashing with non-negative tensor factorization (NTF) for copy detection. In the presented video hashing scheme, secondary frames are computed from the preprocessed video by assigning weights to all frames within a video group based on color entropy. Next, the secondary frames are fed into the pre-trained MobileNetV2 and then NTF is exploited to compress the three-order tensor constructed by stacking the output feature maps for hash construction. Experiments conducted on publicly available video datasets indicate that the presented hashing scheme outperforms the evaluated hashing schemes in the performances of classification and copy detection.

Abstract This paper proposes a novel video hashing with tensor robust Principal Component Analysis (PCA) and Histogram of Optical Flow (HOF) for copy detection. In the proposed hashing, a video is divided into some video groups. For each video group, a low-rank secondary frame is constructed from the low-rank component decomposed by applying tensor robust PCA to the video group. Since the low-rank component can well indicate spatial-temporal intrinsic structure of the video group and it is slightly disturbed by digital operations, feature extraction from the low-rank secondary frames is discriminative and stable. Next, spatial features and temporal features are extracted from low-rank secondary frames by Charlier moments and HOF, respectively. Since the Charlier moments are robust to geometric transform and they can efficiently distinguish video frames with different contents, the use of Charlier moments can make robust and discriminative spatial features. As the HOF can measure the distribution of motion information between frames, the temporal features formed by HOFs can provide good discrimination. Hash is ultimately determined by quantizing the spatial and temporal features and concatenating the quantized results. Numerous experiments on open video datasets indicate that the proposed hashing is superior to some hashing baseline schemes in terms of classification and copy detection.

Hashing is an efficient technology for various image tasks. This article proposes an effective image hashing with deep and moment features for content authentication. The deep features are calculated by Wavelet scattering network (ScatNet) and local tangent space alignment (LTSA). The ScatNet is used to construct a third-order tensor from the image brightness component in the polar coordinates transformation (PCT) domain and the LTSA is used to learn the compact features from the third-order tensor. The moment features are contributed by the tchebichef moments (TMs) and quaternion bessel fourier moments (QBFMs), where the TMs can measure shape features and the QBFMs can reflect color features. Extensive experiments on four public databases are done to verify performances of the proposed algorithm. The results demonstrate that the proposed algorithm is superior to some baseline algorithms in content authentication.

Robust hashing is a useful technique for the image applications of watermarking, authentication, quality assessment and copy detection. This paper proposes a new robust hashing for image copy detection by using local tangent space alignment (LTSA). A key contribution is the weighted visual map computation based on the difference of Gaussian (DOG) and visual attention model. The weighted visual map can provide the proposed method with good robustness. Another contribution is the feature learning via LTSA from the feature matrix of the weighted visual map in discrete cosine transform domain. As it can maintain the local geometric relationships within image, the learned features can make the proposed method discriminative. Extensive experiments on public databases are conducted to validate the proposed robust hashing method. Compared with some famous robust hashing methods, the proposed robust hashing method demonstrates preferable classification performance in terms of discrimination and robustness. Copy detection performance is tested and the result verifies effectiveness of the proposed robust hashing method.

Copy detection is a key task of image copyright protection. This article proposes a robust image hashing algorithm by CP decomposition and discrete cosine transform (DCT) for copy detection. The first contribution is the third-order tensor construction with low-frequency coefficients in the DCT domain. Since the low-frequency DCT coefficients contain most of the image energy, they can reflect the basic visual content of the image and are less disturbed by noise. Hence, the third-order tensor construction with the low-frequency DCT coefficients can ensure robustness of our algorithm. Another contribution is the application of the CP decomposition to the third-order tensor for learning a short binary hash. As the factor matrices learned from the CP decomposition can preserve the topology of the original tensor, the binary hash derived from the factor matrices can reach good discrimination. Lots of experiments and comparisons are done to validate effectiveness and advantage of our algorithm. The results demonstrate that our algorithm has superior classification and copy detection performances than several baseline algorithms. In addition, our algorithm is also better than some baseline algorithms with regard to hash length and computational time.

Robust hashing is a powerful technique for processing large-scale images. Currently, many reported image hashing schemes do not perform well in balancing the performances of discrimination and robustness, and thus they cannot efficiently detect image copies, especially the image copies with multiple distortions. To address this, we exploit global and local invariant features to develop a novel robust hashing for image copy detection. A critical contribution is the global feature calculation by gray level co-occurrence moment learned from the saliency map determined by the phase spectrum of quaternion Fourier transform, which can significantly enhance discrimination without reducing robustness. Another essential contribution is the local invariant feature computation via Kernel Principal Component Analysis (KPCA) and vector distances. As KPCA can maintain the geometric relationships within image, the local invariant features learned with KPCA and vector distances can guarantee discrimination and compactness. Moreover, the global and local invariant features are encrypted to ensure security. Finally, the hash is produced via the ordinal measures of the encrypted features for making a short length of hash. Numerous experiments are conducted to show efficiency of our scheme. Compared with some well-known hashing schemes, our scheme demonstrates a preferable classification performance of discrimination and robustness. The experiments of detecting image copies with multiple distortions are tested and the results illustrate the effectiveness of our scheme.

Copy detection is a key task of image copyright protection. Most robust hashing schemes do not make satisfied performance of image copy detection yet. To address this, a robust hashing scheme with deep features and Meixner moments is proposed for image copy detection. In the proposed hashing, global deep features are extracted by applying tensor Singular Value Decomposition (t-SVD) to the three-order tensor constructed in the DWT domain of the feature maps calculated by the pre-trained VGG16. Since the feature maps in the DWT domain are slightly disturbed by digital operations, the constructed three-order tensor is stable and thus the desirable robustness is guaranteed. Moreover, since t-SVD can decompose a three-order tensor into multiple low-dimensional matrices reflecting intrinsic structure, the global deep feature calculation from the low-dimensional matrices can provide good discrimination. Local features are calculated by the block-based Meixner moments. As the Meixner moments are resistant to geometric transformation and can efficiently discriminate various images, the use of the block-based Meixner moments can make discriminative and robust local features. Hash is ultimately determined by quantifying and combining global deep features and local features. The results of extensive experiments on open image datasets demonstrate that the proposed robust hashing outperforms some state-of-the-art robust hashing schemes in terms of classification and copy detection performances.

Abstract Image hashing is an effective technology for extensive image applications, such as retrieval, authentication and copy detection. This paper designs a new image hashing scheme based on saliency map and sparse model. The major contributions are twofold. The first contribution is the construction of a weighted image representation by combining a visual attention model called Itti model and the matrix of color vector angle (CVA). Since the Itti model can efficiently detect saliency map and CVA fully captures color information of image, they contribute to a visually robust and discriminative image representation. The second contribution is the hash extraction from the weighted image representation via sparse model. A classical sparse model called robust principal component analysis is exploited to decompose the weighted image representation into a low-rank component and a sparse component. As the low-rank component can describe intrinsic structure of image, hash calculation with low-rank component can achieve good discrimination. The efficiencies of the proposed scheme are validated by extensive experiments with open databases. The results demonstrate that the proposed scheme is superior to some state-of-the-art schemes in terms of classification performance between robustness and discrimination.

Image copy detection is an important technology of copyright protection. This paper proposes an efficient hashing method for image copy detection using 2D-2D (two-directional two-dimensional) PCA (Principal Component Analysis). The key is the discovery of the translation invariance of 2D-2D PCA. With the property of translation invariance, a novel model of extracting rotation-invariant low-dimensional features is designed by combining PCT (Polar Coordinate Transformation) and 2D-2D PCA. The PCT can convert an input rotated image to a translation matrix. Since the 2D-2D PCA is invariant to translation, the low-dimensional features learned from the translation matrix are rotation-invariant. Moreover, vector distances of low-dimensional features are stable to common digital operations and thus hash construction with the vector distances is of robustness and compactness. Three open image datasets are exploited to conduct various experiments for validating efficiencies of the proposed method. The results demonstrate that the proposed method is much better than some representative hashing methods in the performances of classification and copy detection.

Compression technology for representing image is on demand for efficiently processing images in the Big Data era. Image hashing is an effective compression technology for computing a short representation based on visual content of input image. Currently, most reported image hashing algorithms have weakness in making a desirable classification between discrimination and robustness and thus can not reach good performance in copy detection. To address these issues, this paper proposes a new robust image hashing with Isometric Mapping (Isomap) and saliency map for copy detection. A key contribution is hash generation with saliency map determined by the Frequency Tuned (FT) method, which can guarantee robustness of the proposed image hashing. Another contribution is the use of Isomap in deriving hash from the FT-based saliency map. Since Isomap can discover the internal geometry features of image, the use of Isomap can learn discriminative image features and thus discrimination of the proposed image hashing is ensured. Experiments on open image databases are carried out. Comparison results illustrate that the proposed image hashing is better than some state-of-the-art algorithms in the performances of classification and copy detection.

Image quality assessment (IQA) is an important task of computer vision. Most full-reference (FR) IQA methods do not reach desirable prediction performance. To address this issue, we propose a novel multi-level feature aggregation network (MLFAN) for FR-IQA. An important contribution is an effective multi-level feature aggregation network. This network utilizes a siamese network with vision transformer for multi-level feature extraction. It compares images at the multi-level perceptual feature differences by considering the relationship among color, texture and shape information, focuses more on the salient regions by an attention aggregator and scores images by a two-branch prediction head. Another important contribution is a novel loss function. This loss function jointly utilizes Mean Square Error, KL divergence and rank order of quality scores to provide stable training. It makes the proposed MLFAN-IQA method effectively learn perceptual quality of images. Experiments are done to test IQA performance of the proposed MLFAN-IQA method. Comparisons show that the proposed MLFAN-IQA method outperforms some state-of-the-art FR-IQA methods on the datatsets of conventional distorted images. Moreover, the proposed MLFAN-IQA method also reaches comparable performance on the dataset of GAN-based synthetic distorted images.

Abstract Hashing scheme is a high‐efficiency technique for processing massive images. Two critical metrics of the hashing scheme are discrimination and robustness, but most schemes do not get satisfied classification performance between them. This paper proposes a novel hashing scheme via image feature map and 2D PCA. First, the proposed scheme extracts local phase quantization (LPQ) features in the frequency domain and local ternary pattern (LTP) features in the spatial domain, and combines them to construct an image feature map. Second, the proposed scheme conducts dimension reduction via 2D PCA for learning features from the image feature map. Last, the learned features are compressed to generate the hash sequence. Performances are tested on open image datasets. The results demonstrate that the proposed scheme can make a good balance between discrimination and robustness. In addition, the classification and copy detection of the proposed scheme are both superior to those of some famous hashing schemes.

Reversible semi-fragile authentication watermark (RSAW) is required in an integrated and powerful authentication system. An effective RSAW scheme should have the desirable features: tamper detection and localization, good perceptual invisibility, detection without requiring explicit knowledge of the original image, robustness against lossy compression, noise attack and low-pass filtering to some extent, reversibility on condition that marked image has not been disturbed, and high security against forge attack. To our best knowledge, RSAW schemes that are presented in the literature are not effective enough. This paper proposes a new RSAW scheme which is effective and has additional features as tamper discerning, computational efficiency and multiple encryption keys supporting. Experimental results demonstrate the validity of the proposed RSAW scheme.