Combining multiple classifiers is a promising approach to hardening forensic detectors against adversarial evasion attacks. The key idea is that an attacker must fool all individual classifiers to evade detection. The 1.5C classifier is one of these multiple-classifier detectors that is attack-agnostic, and thus even increases the difficulty for an omniscient attacker. Recent work evaluated the 1.5C classifier with SPAM features for image manipulation detection. Despite showing promising results, their security analysis leaves several aspects unresolved. Surprisingly, the results reveal that fooling only one component is often sufficient to evade detection. Additionally, the authors evaluate classifier robustness with only a black-box attack because, currently, there is no white-box attack against SPAM feature-based classifiers. This paper addresses these shortcomings and complements the previous security analysis. First, we develop a novel white-box attack against SPAM feature-based detectors. The proposed attack produces adversarial images with lower distortion than the previous attack. Second, by analyzing the 1.5C classifier’s acceptance region, we identify three pitfalls that explain why the current 1.5C classifier is less robust than a binary classifier in some settings. Third, we illustrate how to mitigate these pitfalls with a simple axis-aligned split classifier. Our experimental evaluation demonstrates the increased robustness of the proposed detector for SPAM feature-based image manipulation detection.

Learning-based algorithms for automated license plate recognition implicitly assume that the training and test data are well aligned. However, this may not be the case under extreme environmental conditions, or in forensic applications where the system cannot be trained for a specific acquisition device. Predictions on such out-of-distribution images have an increased chance of failing. But this failure case is oftentimes hard to recognize for a human operator or an automated system. Hence, in this work we propose to model the prediction uncertainty for license plate recognition explicitly. Such an uncertainty measure allows to detect false predictions, indicating an analyst when not to trust the result of the automated license plate recognition. In this paper, we compare three methods for uncertainty quantification on two architectures. The experiments on synthetic noisy or blurred low-resolution images show that the predictive uncertainty reliably finds wrong predictions. We also show that a multi-task combination of classification and super-resolution improves the recognition performance by 109% and the detection of wrong predictions by 29%.

Identifying the model of a camera that has captured an image can be an important task in criminal investigations. Many methods assume that the image under analysis originates from a given set of known camera models. In practice, however, a photo can come from an unknown camera model, or its appearance could have been altered by unknown post-processing. In such a case, forensic detectors are prone to fail silently. One way to mitigate silent failures is to use a rejection mechanism for unknown examples. In this work, we propose Gaussian processes (GPs), which intrinsically provide such a rejection mechanism. This makes GPs a potentially powerful tool in multimedia forensics, where forensic analysts regularly work on images from unknown origins. We demonstrate that GPs scale well to the task of camera model identification. Probabilistic predictions from a GP classifier achieve high classification accuracy for known camera models while providing reliable uncertainty estimates. The built-in uncertainty estimates effectively tackle open-set camera model identification, outperforming two state-of-the-art methods.

In multimedia forensics, learning-based methods provide state-of the-art performance in determining origin and authenticity of images and videos. However, most existing methods are challenged by out-of-distribution data, i.e., with characteristics that are not covered in the training set. This makes it difficult to know when to trust a model, particularly for practitioners with limited technical background.In this work, we make a first step toward redesigning forensic algorithms with a strong focus on reliability. To this end, we propose to use Bayesian neural networks (BNN), which combine the power of deep neural networks with the rigorous probabilistic formulation of a Bayesian framework. Instead of providing a point estimate like standard neural networks, BNNs provide distributions that express both the estimate and also an uncertainty range.We demonstrate the usefulness of this framework on a classical forensic task: resampling detection. The BNN yields state-of-the-art detection performance, plus excellent capabilities for detecting out-of-distribution samples. This is demonstrated for three pathologic issues in resampling detection, namely unseen resampling factors, unseen JPEG compression, and unseen resampling algorithms. We hope that this proposal spurs further research toward reliability in multimedia forensics.

Image reconstruction is particularly difficult when the type of image degradations are unknown. This may be the case if the acquisition device is unknown or the images stem from an uncontrolled environment like the internet. Yet, it may be important to reconstruct a specific piece of information from the image, such as digits from signs or vehicle license plates. Existing works incorporate such prior information with a sequential super-resolution and classification pipeline. However, this approach is prone to error propagation.In this work, we propose a new approach of connecting classification and super-resolution in parallel within a multi-task network. We show that this architecture is able to preserve structures and to remove noisy pixels although the network itself has never been trained on noisy data. We also show that this design allows to transparently trade classification and super-resolution quality. On upsampling by factor 4, we outperform sequential approaches in terms of SSIM by 10% and improve classification by 69%.

The JPEG compression format provides a rich source of forensic traces that include quantization artifacts, fingerprints of the container format, and numerical particularities of JPEG compressors. Such a diverse set of cues serves as the basis for a forensic examiner to determine origin and authenticity of an image. In this work, we present a novel artifact that can be used to fingerprint the JPEG compression library. The artifact arises from chroma subsampling in one of the most popular JPEG implementations. Due to integer rounding, every second column of the compressed chroma channel appears on average slightly brighter than its neighboring columns, which is why we call the artifact a "chroma wrinkle". We theoretically derive the chroma wrinkle footprint in DCT domain, and use this footprint for detecting chroma wrinkles. The artifact is detected with more than 90% accuracy on images of JPEG quality 75 and above. Our experiments indicate that the artifact can also be used for manipulation localization, and that it is robust to several global postprocessing operations.