Most data-driven deep learning-based Artistic Image Aesthetics Assessment (AIAA) methods cannot effectively extract visual attributes from art images since the existing artistic image datasets don’t provide any information about visual attributes. The lack of visual attributes reduces the interpretability of AIAA methods and limits their performance. To address these problems, a novel artistic image aesthetics assessment assisted by photographic visual attributes is proposed. The proposed method consists of a feature extraction module and a joint prediction module. The feature extraction module pre-trained on a photographic dataset and an artistic image dataset can learn the information of photographic attributes and the generic artistic aesthetic information. The joint prediction module uses a non-local self-attention block to fuse the photographic visual attribute features with general artistic aesthetic features. The fused features are fed into an FC layer for calculating the artistic image aesthetic score. Experimental results indicate that our proposed method outperforms some state-of-the-art AIAA methods.

Fine-grained image retrieval (FGIR) is an important topic of image retrieval, and its challenge lies in the accurate identification of image objects with minor inter-class differences and considerable intraclass differences. Most existing methods exploit Convolutional Neural Networks (CNNs) to capture fine-grained and coarse-grained information while overlooking the scale variations. To address these issues, a novel method named Hash Generation Network (HGNet) guided by high frequency information is developed to learn crucial details across different scales. The HGNet consists of a High-Frequency Guidance Module (HFGM) and a Hash Generation Module (HGM). The key contribution is the proposed HFGM which integrates the high-frequency information and multi-scale features extracted from the Swin Transformer. As the Swin Transformer can effectively capture global contextual information, its multi-scale features, guided by high-frequency information that contains fine-grained texture details, can represent both fine-grained and coarse-grained details, thereby guiding the HGM in generating discriminative hash codes. Experimental results show that the HGNet outperforms several SOTA FGIR methods in retrieval performance.

No-Reference Image Quality Assessment (NR-IQA) is an important task of computer vision. Most deep neural networks based NR-IQA methods have the ability of accurate quality predictions, but they have large-scale parameters and high computational complexity. To alleviate these problems, we propose a lightweight NR-IQA method by unifying statistical and refined semantic features. Our proposed method consists of a lightweight feature extractor, a Statistical Semantic Feature Extraction (SSFE) module, and a Refined Semantic Feature Extraction (RSFE) module. The lightweight feature extractor is used to extract semantic features with perceptual distortion information. The SSFE module is designed to obtain statistical information of the semantic features for capturing the local and global changes of distorted image. The RSFE module is designed to refine the semantic features for measuring complex distortions. Extensive experiments on many IQA datasets are done and the results indicate that our proposed method outperforms some baseline NR-IQA methods in IQA performance, generalization ability, and model complexity.

Image aesthetics assessment is commonly treated as a classification or regression task and its performance bottleneck mainly depends on the effective utilization of aesthetic features. Traditional methods tend to use only subjective or objective features. Only one type of feature overlooks the inherent unity of these features in aesthetic characteristics. To comprehensively utilize image aesthetic characteristics, we propose an innovative image aesthetics assessment method that integrates subjective and objective features. Specifically, the proposed method comprises two modules: feature extraction module and feature fusion module. In the feature extraction module, a dual-branch neural network is proposed to extract subjective and objective aesthetic features of the image. In the feature fusion module, a feature fusion module based on a deep feedforward neural network is proposed to combine these two types of features and generate the final image aesthetic score. The experimental results indicate that our method gets a high performance in aesthetics assessment.

In real scenarios, videos are usually corrupted by multiple types of noise, which brings great challenges to retrieving social videos. However, most of the current video hashing methods for video retrieval consider the attack of a single noise model, and rarely discuss when dealing with complex noise models, which is not conducive to solving the above difficulties. Thus, we describe a novel video hashing with secure anti-noise model (SANM). To improve the robustness of noise attacks, the input video is reconstructed into a SANM by low-rank representation (LRR) and random subspace partition (RSP). LRR is useful technique for capturing the global structure of data. It focuses on recovering the underlying subspace in noisy environment and helps to make the proposed model robust to multiple noises. In addition, using chaotic mapping to control the generation of RSP can ensure the security of proposed model. Then, a new subspace decomposition descriptor (SDD) is proposed. SDD is obtained by calculating the invariant distances of the factor matrices obtained by tucker decomposition, and is used to decompose SANM to derive a compact hash. Various experiments demonstrate that the SANM hashing performs better than several state-of-the-art algorithms in terms of good robustness and discrimination, and it can accurately retrieve social videos.

Hashing is an efficient technology for various image tasks. This article proposes an effective image hashing with deep and moment features for content authentication. The deep features are calculated by Wavelet scattering network (ScatNet) and local tangent space alignment (LTSA). The ScatNet is used to construct a third-order tensor from the image brightness component in the polar coordinates transformation (PCT) domain and the LTSA is used to learn the compact features from the third-order tensor. The moment features are contributed by the tchebichef moments (TMs) and quaternion bessel fourier moments (QBFMs), where the TMs can measure shape features and the QBFMs can reflect color features. Extensive experiments on four public databases are done to verify performances of the proposed algorithm. The results demonstrate that the proposed algorithm is superior to some baseline algorithms in content authentication.

Abstract Video hashing is a popular research topic in the fields of multimedia information and security because its fast matching and low-cost storage characteristics are widely used in many applications (video copy detection, video retrieval, video authentication, etc.). This paper describes a compact video hashing method with a rotation- and translation-invariant model (RTIM). The key contribution of this approach is that it innovatively reconstructs an input video into a 3D RTIM by combining ring partition and a pipeline histogram; this is a first in video hashing and helps make video hashes resistant to rotation and translation. Then, the proposed model is decomposed via Tucker decomposition, and the generated core tensor is used to produce a compact hash. As the core tensor is a compressed version of the original tensor, hash construction with the core tensor makes RTIM hashing compact and achieves desirable discrimination ability. Different from existing video hashing algorithms, RTIM hashing can not only resist many commonly used digital operations, especially video rotation and cyclic frame shifting, but also achieve good discrimination ability. Various experiments demonstrate the effectiveness of our algorithm. Receiver operating characteristic curve comparisons show that compared with the state-of-the-art video hashing algorithms, RTIM hashing is more robust and compact.

Convolutional neural networks (CNNs) have been widely applied to motor fault diagnosis. However, to obtain high recognition accuracy, massive training data are typically required and transmitted to the cloud/local server for training, which may suffer from security and privacy problems. In this article, a noise-boosted CNN (NBCNN) model is developed to achieve accelerated training and improved recognition accuracy with limited training samples. First, the NBCNN model with a noise-injection fully connected layer is established. Then, a strategy for noise selection and injection is proposed to obtain an optimal matching among the data, model, and noise. Finally, the optimal injected noise accelerates the convergence of model training and improves the accuracy of motor fault diagnosis. Compared with the conventional CNN without noise injection and the state-of-the-art models, the effectiveness and superiority of the proposed NBCNN model are validated by two benchmark datasets. In addition, the algorithm is deployed onto an edge device and the results show that the training speed of the developed NBCNN can reach nine times faster than the conventional CNN. The proposed method shows remarkable potential for distributed model training, federal learning, and real-time motor fault diagnosis.

With the rapid development of deepfake techniques, massive face manipulation videos appeared on social networks. These deepfake videos not only violated the original video of the author’s privacy, but also seriously threatened the security of the video database. Robust video hashing can map videos with similar visual content into similar hash codes, which is beneficial for tracking fake video material in social networks. In this paper, a robust video hashing algorithm based on local fluctuation preserving is proposed. The algorithm uses a shot segmentation model and local statistical descriptors, which is robust to many commonly-used digital operations and can accurately track the original version of these fake videos from a huge video database. An essential contribution is a shot segmentation model reconstruction from input video with image hashing and discrete wavelet transform, reaching initial data compression and against noise attack. In addition, as local statistical descriptors are content-based and local preserving features, the hash generated by local statistical descriptors can achieve good discrimination and ensure that the proposed hash has good tracking ability to fake videos from original videos.

Abstract Video hashing is a novel technique of multimedia processing and finds applications in video retrieval, video copy detection, anti-piracy search and video authentication. In this paper, we propose a robust video hashing based on discrete cosine transform (DCT) and non-negative matrix decomposition (NMF). The proposed video hashing extracts secure features from a normalized video via random partition and dominant DCT coefficients, and exploits NMF to learn a compact representation from the secure features. Experiments with 2050 videos are carried out to validate efficiency of the proposed video hashing. The results show that the proposed video hashing is robust to many digital operations and reaches good discrimination. Receiver operating characteristic (ROC) curve comparisons illustrate that the proposed video hashing outperforms some state-of-the-art algorithms in classification between robustness and discrimination.

This paper presents a new image hashing that is designed with tensor decomposition (TD), referred to as TD hashing, where image hash generation is viewed as deriving a compact representation from a tensor. Specifically, a stable three-order tensor is first constructed from the normalized image, so as to enhance the robustness of our TD hashing. A popular TD algorithm, called Tucker decomposition, is then exploited to decompose the three-order tensor into a core tensor and three orthogonal factor matrices. As the factor matrices can reflect intrinsic structure of original tensor, hash construction with the factor matrices makes a desirable discrimination of the TD hashing. To examine these claims, there are 14,551 images selected for our experiments. A receiver operating characteristics (ROC) graph is used to conduct theoretical analysis and the ROC comparisons illustrate that the TD hashing outperforms some state-of-the-art algorithms in classification performance between the robustness and discrimination.

We propose a novel perceptual image hashing based on weighted discrete wavelet transform (DWT) statistical features. This hashing converts input image into a normalized image by bi-linear interpolation and color space conversion, extracts edge image of the normalized image via Canny operator, and divides the edge image into non-overlapping blocks. For each block, a three-level 2D DWT is applied to obtain different sub-bands and the weighted sum of the DWT statistics of these sub-bands is calculated. Finally, image hash is generated by concatenating and quantizing these weighted DWT features. Similarity of image hashes is measured by Euclidean distance. The Copydays dataset and the Uncompressed Color Image Database (UCID) are both used to evaluate classification between robustness and discrimination. Receiver operating characteristics curve comparisons illustrate that our hashing is superior to some state-of-the-art algorithms in classification performance with respect to robustness and discrimination. The LIVE Image Quality Assessment Database is used to validate our application in reduced-reference image quality assessment. Experimental results show that our hashing has better performance in image quality assessment than two popular measures, i.e. peak signal-to-noise ratio and structural similarity.