Image manipulation localization remains a hot topic due to its inherent semantic-independent nature and realistic needs. Virtually all localization studies are devoted to solving arbitrary tampering using multi-branch networks based on deep features or skip-connection structures based on full features, which may induce the loss of manipulation details or noisy interference from image semantics. This poses a challenge for existing localization methods to fully capture invisible manipulations, especially in post-processing settings and across dataset scenarios. To address the above issues, we propose an uncertainty-supervised parallel network (UP-Net) for image tampering localization that preserves more manipulation details while avoiding semantic noise. UP-Net cascades the frequency and RGB domains of the manipulated image as dual-domain embedding, instead of dual-domain parallel learning as in previous work. To learn semantic-independent manipulation features, two structurally identical parallel branches are designed to learn tampering inconsistencies from intermediate and deep coding features for gradually obtaining the initial and final localization predictions. Where attention-guided partial decoder (AGPD) integrates more precise manipulation edges and manipulation semantics without introducing additional noise by focusing on channel correlation and spatial dependence, making a significant contribution to performance. Moreover, the new concept of uncertainty-constrained loss supervision is introduced to guide UP-Net to continuously improve confidence in locating difficult pixels, which are easily misclassified due to post-processing operations. Experiments on three public manipulation datasets and two real challenge datasets show that our end-to-end UP-Net achieves significant performance in manipulation localization, generalization across datasets, and robustness compared to state-of-the-art methods.

Human motion prediction aims at capturing the hidden temporal correlations between historical motion and future poses. Various graph convolution networks have been presented for encoding the spatial dependencies between joints. Empirically, the crucial shortcoming of these methods is that they fail to extract enough spatially relevant information. In this paper, we propose an adaptive multi-order context fusion architecture that consists of two components. A novel message propagation module encodes the interaction between joints, while highlighting contexts from closely related joints. An adaptive aggregation module fuses various information from different-order joint features. Our model is evaluated on Human 3.6 Million dataset. Extensive experiments show that our method achieves state-of-the-art performance on short-term and long-term predictions.

Deep learning has become an emerging technical for image manipulation localization, which can automatically recognize abnormal traces caused by manipulation. However, as manipulations mainly happens in the foreground regions, these methods largely focus on the foreground contents and neglect the background, which contain complementary signal for fully understanding the image and are meaningful for manipulation localization. We propose a Mutually-Complementary Network (MC-Net), which is a two-branch network to operate the foreground and background features, respectively. To distill complementary signals from the features, we propose a mutual attentive module composed of self-feature attentive, and cross-feature attentive components to advance the communication across the foreground and background branches. Extensive qualitative and quantitative experiments demonstrate that our proposed MC-Net distinctly improves the prediction of foreground and background, obtains consistent performance increments on four benchmark data sets, and significantly outperforms the state-of-the-art methods.

Predicting human motion from a historical pose sequence is at the core of many applications in computer vision. Current state-of-the-art methods concentrate on learning motion contexts in the pose space, however, the high dimensionality and complex nature of human pose invoke inherent difficulties in extracting such contexts. In this paper, we instead advocate to model motion contexts in the joint trajectory space, as the trajectory of a joint is smooth, vectorial, and gives sufficient information to the model. Moreover, most existing methods consider only the dependencies between skeletal connected joints, disregarding prior knowledge and the hidden connections between geometrically separated joints. Motivated by this, we present a semi-constrained graph to explicitly encode skeletal connections and prior knowledge, while adaptively learn implicit dependencies between joints.We also explore the applications of our approach to a range of objects including human, fish, and mouse. Surprisingly, our method sets the new state-of-the-art performance on 4 different benchmark datasets, a remarkable highlight is that it achieves a 19.1% accuracy improvement over current state-of-the-art in average. To facilitate future research, we have released our code at https://github.com/Pose-Group/MPT.

Motion prediction is a classic problem in computer vision, which aims at forecasting future motion given the observed pose sequence. Various deep learning models have been proposed, achieving state-of-the-art performance on motion prediction. However, existing methods typically focus on modeling temporal dynamics in the pose space. Unfortunately, the complicated and high dimensionality nature of human motion brings inherent challenges for dynamic context capturing. Therefore, we move away from the conventional pose based representation and present a novel approach employing a phase space trajectory representation of individual joints. Moreover, current methods tend to only consider the dependencies between physically connected joints. In this paper, we introduce a novel convolutional neural model to effectively leverage explicit prior knowledge of motion anatomy, and simultaneously capture both spatial and temporal information of joint trajectory dynamics. We then propose a global optimization module that learns the implicit relationships between individual joint features. Empirically, our method is evaluated on large-scale 3D human motion benchmark datasets (i.e., Human3.6M, CMU MoCap). These results demonstrate that our method sets the new state-of-the-art on the benchmark datasets. Our code is released at https://github.com/Pose-Group/TEID.

The method proposes a multi-focus noisy image fusion algorithm combining gradient regularized convolutional sparse representatione and spatial frequency. Firstly, the source image is decomposed into a base layer and a detail layer through two-scale image decomposition. The detail layer uses the Alternating Direction Method of Multipliers (ADMM) to solve the convolutional sparse coefficients with gradient penalties to complete the fusion of detail layer coefficients. Then, The base layer uses the spatial frequency to judge the focus area, the spatial frequency and the "choose-max" strategy are applied to achieved the multi-focus fusion result of base layer. Finally, the fused image is calculated as a superposition of the base layer and the detail layer. Experimental results show that compared with other algorithms, this algorithm provides excellent subjective visual perception and objective evaluation metrics.

In this study, an improved quantum‐behaved particle swarm optimisation based pulse‐coupled neural network (IQPSO‐PCNN) is proposed in the non‐subsampled shearlet transform (NSST) domain for medical image fusion. First, NSST tool is used to decompose the source image into low‐frequency and high‐frequency subbands. Then, for low‐frequency subbands, the fusion rules of two different functions are presented, which simultaneously addresses two key issues of energy preservation and detail extraction. For high‐frequency subbands, unlike conventional PCNN‐based methods, parameters are manually set based on experience, and the decomposed high‐frequency subbands share a set of parameters. The IQPSO‐PCNN model can obtain the optimal parameters for each high‐frequency subband adaptively according to its own information. Finally, the fused low‐frequency subband and high‐frequency subbands are inversely transformed by NSST to acquire the final fused image. The proposed algorithm uses >90 pairs of images with four different modalities. In addition, fusion experiments are performed on different sequences of the three modes. The experimental results demonstrate that the proposed method is superior to existing state‐of‐art methods in subjective visual performance and objective evaluation.

With the rapid development of digital phones, the digital image forensics system in current times has had a great impact. It will lead to a serious threat for us, and especially the emergence of the recaptured image makes the existing digital image forensics algorithm invalid. So, it needs an effective image detection algorithm for us to identify recaptured images. In this paper, a new detection algorithm of the recaptured image is presented based on gray level co-occurrence matrix by analyzing the differences between the real and recaptured images. In order to analyze the differences, a new image evaluation model was put forward in this paper, which is called image variance ratio. Firstly, the algorithm proposed extracted high-frequency and low-frequency information of images by wavelet transform, based on which we calculated the relative gray level co-occurrence matrices. Secondly, the features of gray level co-occurrence matrix were extracted. At last, the recaptured image was classified by the support vector machine according to the features. The experimental results showed the algorithm proposed can not only effectively identify the recaptured image obtained from different media but also have better identification rate.

We propose a multi-focus image fusion method, in which a fully convolutional network for focus detection (FD-FCN) is constructed. To obtain more precise focus detection maps, we propose to add skip layers in the network to make both detailed and abstract visual information available when using FD-FCN to generate maps. A new training dataset for the proposed network is constructed based on dataset CIFAR-10. The image fusion algorithm using FD-FCN contains three steps: focus maps are obtained using FD-FCN, decision map generation occurs by applying a morphological process on the focus maps, and image fusion occurs using a decision map. We carry out several sets of experiments, and both subjective and objective assessments demonstrate the superiority of the proposed fusion method to state-of-the-art algorithms.

This letter focuses on image manipulation detection which aims to recognize the manipulated regions under the contextual semantic information. Existing approaches usually overlook the semantic discrepancy between different levels of feature maps, and directly fuse (e.g., addition, or concatenation) them for detection. In this letter, we argue that the semantic gap is the main reason for the low effectiveness of feature fusion in manipulation predictions. To address this problem, we propose a Global Semantic Consistency Network (GSCNet) for image manipulation detection, which is based on an encoder-decoder structure. Specifically, to make GSCNet include more global texture information which has been empirically confirmed to be beneficial to manipulation detection, gram block is first deployed on each level of feature maps in the encoding stage. Based on that, bi-directional convolutional LSTM is further implemented on the decoding stage, such that feature maps of the same level have semantic consistency. Experimental results on NIST16, and CASIA v1.0 declare that GSCNet can accurately locate the manipulated regions. Furthermore, compared to the existing models, GSCNet can achieve new state-of-the-art results.

To improve the poor robustness and low accuracy of the existing algorithms of image splicing detection, a novel passive image forgery detection method is proposed in this study, which is based on DOCT (discrete octonion cosine transform) and Markov. By introducing the octonion and DOCT, the colour information of six image channels (the RGB model and the HSI model) can be exhaustively extracted, which enhances the robustness of the algorithm. On the issue of improving the detection accuracy, the standard deviation is used to characterise the relationship of the colour information between the parts of DOCT coefficient matrix, and the K ‐fold cross‐validation is introduced to improve the identification performance of the classifier. The steps of the algorithm are as follows: Firstly, the 8 × 8 block DOCT transform is used to the original image to obtain parts of block DOCT coefficient. Secondly, the standard deviation is used to process the corresponding parts of all blocks of the image. Finally, the Markov feature vector of the DOCT coefficient is extracted and feds to the LIBSVM (a library for support vector machines). When using LIBSVM for classification, K ‐fold cross‐validation is executed to select the best parameter pairs. The experiment results demonstrate that the algorithm is superior to the other state‐of‐the‐art splicing detection methods.

With the rapid development of digital cameras and smart phones, the image identification system in current times will be of a great impact. This will cause the form of image information to increase serious security issues. Especially, the emergence of the recaptured image makes conventional digital image forensics algorithm invalid. Therefore, a new image forensics algorithm is urgently needed to identify the recaptured image. In this paper, a new recaptured image identifying algorithm is put forward based on wavelet transformation and noise analysis by analyzing the differences between the real and recaptured images generated in the imaging process. First, the proposed algorithm extracts mean value, variance and skewness as wavelet characteristic from the high-frequency images and low-frequency images by wavelet transformation. Meanwhile, the proposed algorithm analyzes the noise image by means of local binary pattern to extract noise characteristic. Finally, the support vector machine is applied to classify the recaptured image with wavelet characteristics and noise characteristics. The results show the presented method can not only identify the recaptured image obtained from different media but also have better identification rate, and the dimension of the characteristic vector is also lower than those obtained by other algorithms.

To further improve the detection rate with relatively low dimension feature vector, a novel passive splicing detection method using textural features based on the grey level co‐occurrence matrices, namely TF‐GLCM, is proposed in this study. In the TF‐GLCM, the GLCM are calculated based on the difference block discrete cosine transform arrays to capture the textural information and the spatial relationship between image pixels sufficiently. The discriminable properties contained in the GLCM are described by six textural features, which include two new introduced ones and four independent ones. In addition, the statistical moments mean Me and standard deviation SD of textural features are used instead of themselves as elements in feature vector to reduce the dimensionality of feature vector and computational complexity. A support vector machine is employed for classification purpose. Experimental results show that the TF‐GLCM achieves the detection rates of 98% on CASIA v1.0, and 97% on CASIA v2.0 with 96‐D feature vector. The detection rates benefit from the two new textural features. Meanwhile, the TF‐GLCM is superior to some state‐of‐the‐art methods with lower dimension feature vector.

We present COVERAGE - a novel database containing copy-move forged images and their originals with similar but genuine objects. COVERAGE is designed to highlight and address tamper detection ambiguity of popular methods, caused by self-similarity within natural images. In COVERAGE, forged-original pairs are annotated with (i) the duplicated and forged region masks, and (ii) the tampering factor/similarity metric. For benchmarking, forgery quality is evaluated using (i) computer vision-based methods, and (ii) human detection performance. We also propose a novel sparsity-based metric for efficiently estimating forgery quality. Experimental results show that (a) popular forgery detection methods perform poorly over COVERAGE, and (b) the proposed sparsity based metric best correlates with human detection performance. We release the COVERAGE database to the research community.

A dynamic target recognition system is described which uses blackboard structure to organize the knowledge of images targets, background objects, and procedures. An optical matcher is used to find a suitable knowledge source during the process. This system can quickly recognize an incomplete image target.>