Attributing synthetic images to their source generative models is critical for digital forensics and security. While most existing attribution methods can distinguish images produced by known models and reject those from unknown ones, they are unable to verify whether a given image was produced by a specific, previously unseen model. To address this limitation, we formulate an open-set verification problem: determining whether a given image was generated by a specific model. Our key insight is that synthetic images from different models show consistent, content-independent fingerprints in their amplitude spectrum. Based on this insight, we design a dynamic fingerprint simulator capable of simulating over 1.6 trillion generative model architectures. We further train an extractor to capture model-specific fingerprint representations with supervised contrastive learning, enabling accurate attribution of synthetic images, even from previously unseen models. Our method does not rely on any synthetic images, instead, it is trained solely on real images. On DMDetection and AIGCBenchmark, which comprises dozens of state-of-the-art and in-the-wild generative models, our method improves the attribution performance (AUC) of the prior method from random level to 94.05% and 83.05%, respectively. On GenImage and OSMA datasets, we obtain 85.08%, and 88.48% OSCR, outperforming the SOTA methods by 4.30% and 9.37% under the same settings.

Abstract Partially manipulated images pose a growing threat to the reliability of online content. The rapid spread of diffusion-based inpainting tools has made the creation of such manipulations increasingly easy to perform. As a result, the multimedia forensics community is disadvantaged compared to the attackers, as developing effective localization techniques often requires the creation of large datasets, a resource-intensive process due to the necessary human effort. In this paper, we present Beyond the Brush+ + (BtB++), a fully automated pipeline for generating large-scale datasets of realistic inpainted images. Our experiments demonstrate that BtB++ is both flexible and easily integrates different models and configurations, offering the adaptability required to address evolving models and application scenarios. Moreover, an automatic filtering mechanism ensures quality control by discarding low-quality generated images. To provide an initial assessment of the proposed filtering strategy, we also conducted a small-scale human evaluation, studying the alignment between human perceptual judgments and the automatic metrics used for filtering.

Determining the origin of a digital image or video, namely device source identification, is widely used in courtroom evidence and copyright protection. Currently, device source identification primarily focuses on images captured using single camera with default settings. However, with the advancement of imaging technology, there is a large number of smartphones equipped with multiple cameras and various shooting modes for acquiring images, which may pose a significant challenge to device source identification. Therefore, to assess the performance of image source identification algorithm for modern smartphones and promote further research, it is crucial to build a dataset of image and video captured by modern smartphones. In this paper, we present a large-scale image and video dataset for forensic analysis, ForensiCam-215K. The dataset includes over 215K media contents captured by 130 modern smartphones of 10 major brands. We used the latest equipment to capture images from the main, wide-angle, and telephoto cameras in six different shooting modes, and the media were collected under a strictly controlled procedure to reduce the bias caused by differences in the acquisition process between different devices. Additionally, we used the Photo Response Non-Uniformity (PRNU) method to perform device source identification tests on the dataset. The results indicate that device source identification is a challenging task especially for images and videos captured by smartphones with multiple cameras and various shooting modes. The dataset will be released as open-source and freely available for use by the multimedia forensics research community at https://github.com/dswdsw21072/ForensiCam-215K.

The increasing prevalence of deepfake audio technologies and their potential for malicious use in fields such as politics and media has raised significant concerns regarding the ability to distinguish fake from authentic audio recordings. This study proposes a robust technique for detecting synthetic audio by leveraging three spectral features: Linear Frequency Cepstral Coefficients (LFCC), Mel Frequency Cepstral Coefficients (MFCC), and Constant Q Cepstral Coefficients (CQCC). These features are processed using an enhanced ResNeXt architecture to improve classification accuracy between genuine and spoofed audio. Additionally, a Multi-Layer Perceptron (MLP)-based fusion technique is employed to further boost the model’s performance. Extensive experiments were conducted using three datasets: the ASVspoof 2019 Logical Access (LA) dataset—featuring text-to-speech (TTS) and voice conversion attacks—the ASVspoof 2019 Physical Access (PA) dataset—including replay attacks—and the ASVspoof 2021 LA, PA and DF datasets. The proposed approach has demonstrated superior performance compared to state-of-the-art methods across all three datasets, particularly in detecting fake audio generated by text-to-speech (TTS) attacks. Its overall performance is summarized as follows: the system achieved an Equal Error Rate (EER) of 1.05% and a minimum tandem Detection Cost Function (min-tDCF) of 0.028 on the ASVspoof 2019 Logical Access (LA) dataset, and an EER of 1.14% and min-tDCF of 0.03 on the ASVspoof 2019 Physical Access(PA) dataset, demonstrating its robustness in detecting various types of audio spoofing attacks. Finally, on the ASVspoof 2021 LA dataset the method achieved an EER of 7.44% and min-tDCF of 0.35.

Over the past decade, the proliferation of social networks introduced new challenges in the multimedia forensic field, such as the identification of the originating platform. Significant strides have been made in the characterization of digital images, exploiting features related to the media container and content. Within the realm of videos, several efforts have been directed towards analyzing the container aspect. However, the utilization of content-based features remains limited due to the intricate nature of video encoding. In this paper, we introduce an approach to identify the source social network of a digital video by leveraging codec-based features. For the purpose, we designed a method to extract and efficiently organize detailed information from H.264/AVC-encoded videos based on a bespoke version of the video decoder tool JM. We show how the proposed method can significantly improve the process of determining the source social network, even when confronted with container-based laundering operations, surpassing existing state-of-the-art results.

Detecting the origin of a digital video within a social network is a critical task that aids law enforcement and intelligence agencies in identifying the creators of misleading visual content. In this research, we introduce an innovative method for identifying the original social network of a video, even when the video has been altered through actions like group of frames removal and file container reconstruction. The proposed method takes advantage of the video encoding’s temporal uniformity, leveraging motion vectors to characterize the specific features associated to various social media platforms. Each video is represented by a graph where nodes correspond to macroblocks. These macroblocks are interconnected by following the inter-prediction rules outlined in the H.264/AVC codec standard. Such a structure can be then classified using a graph neural network to predict the platform on which the video has been shared. Experimental results demonstrate that this approach outperforms both codec- and content-based approaches, underscoring the effectiveness of a structural approach in attributing the social media platform from which videos originated.

Abstract The forensic analysis of digital videos is becoming increasingly relevant to deal with forensic cases, propaganda, and fake news. The research community has developed numerous forensic tools to address various challenges, such as integrity verification, manipulation detection, and source characterization. Each tool exploits characteristic traces to reconstruct the video life-cycle. Among these traces, a significant source of information is provided by the specific way in which the video has been encoded. While several tools are available to analyze codec-related information for images, a similar approach has been overlooked for videos, since video codecs are extremely complex and involve the analysis of a huge amount of data. In this paper, we present a new tool designed for extracting and parsing a plethora of video compression information from H.264 encoded files, including macroblocks structure, prediction residuals, and motion vectors. We demonstrate how the extracted features can be effectively exploited to address various forensic tasks, such as social network identification, source characterization, and double compression detection. We provide a detailed description of the developed software, which is released free of charge to enable its use by the research community to create new tools for forensic analysis of video files.

The extensive spread of fake news on social networks is carried out by a diverse range of users, encompassing private individuals, newspapers, and organizations. With widely accessible image and video editing tools, malicious users can easily create manipulated media. They can then distribute this content through multiple fake profiles, aiming to maximize its social impact. To tackle this problem effectively, it is crucial to possess the ability to analyze shared media to identify the originators of fake news. To this end, multimedia forensics research has advanced tools that examine traces in media, revealing valuable insights into its origins. While combining these tools has proven to be highly efficient in creating profiles of image and video creators, it is important to note that most of these tools are not specifically designed to function effectively in the complex environment of content exchange on social networks. In this paper, we introduce the problem of establishing associations between images and their source profiles as a means to tackle the spread of disinformation on social platforms. To this end, we assembled SocialNews, an extensive image dataset comprising more than 12,000 images sourced from 21 user profiles across Facebook, Instagram, and Twitter, and we propose three increasingly realistic and challenging experimental scenarios. We present two simple yet effective techniques as benchmarks, one based on statistical analysis of Discrete Cosine Transform (DCT) coefficients and one employing a neural network model based on ResNet, and we compare their performance against the state of the art. Experimental results show that the proposed approaches exhibit superior performance in accurately classifying the originating user profiles.

The popularity of social networks as primary mediums for sharing visual content has made it crucial for forensic experts to identify the original platform of multimedia content. Various methods address this challenge, but the constant emergence of new platforms and updates to existing ones often render forensic tools ineffective shortly after release. This necessitates the regular updating of methods and models, which can be particularly cumbersome for techniques based on neural networks which cannot quickly adapt to new classes without sacrificing performance on previously learned ones – a phenomenon known as catastrophic forgetting. Recently, researchers aimed at mitigating this problem via a family of techniques known as continual learning. In this paper we study the applicability of continual learning techniques to the social network identification task by evaluating two relevant forensic scenarios: Incremental Social Platform Classification, for handling newly introduced social media platforms, and Incremental Social Version Classification, for addressing updated versions of a set of existing social networks. We perform an extensive experimental evaluation of a variety of continual learning approaches applied to these two scenarios. Experimental results demonstrate that, although Continual Social Network Identification remains a difficult problem, catastrophic forgetting can be significantly mitigated in both scenarios by retaining only a fraction of the image patches from past task training samples or by employing previous tasks prototypes.

Identifying the originating social network of a digital video is considered a relevant task to support law enforcement agencies and intelligence services in tracing producers of deceptive visual contents. Recent advances in video forensics highlighted how the structure of video containers can be extremely effective in determining the social network of provenance. However, current studies do not consider that a malicious user could easily launder the traces of the social network by rebuilding the container without transcoding. In this letter, we propose a method to identify a video’s originating social network, even when the video container structure is completely unreliable. The proposed method exploits the statistics of DCT coefficients to characterize the different social media encoding properties. With this work, we also built and made available over 1000 videos of different provenance (native, manipulated, exchanged through social networks) to aid the forensic community further researching this topic.

The Variation of Prediction Footprint (VPF), formerly used in video forensics for double compression detection and GOP size estimation, is comprehensively investigated to improve its acquisition capabilities and extend its use to video sequences that contain bi-directional frames (B-frames). By relying on a universal rate-distortion analysis applied to a generic double compression scheme, we first explain the rationale behind the presence of the VPF in double compressed videos and then justify the need of exploiting a new source of information such as the motion vectors, to enhance the VPF acquisition process. Finally, we describe the shifted VPF induced by the presence of B-frames and detail how to compensate the shift to avoid misguided GOP size estimations. The experimental results show that the proposed Generalized VPF (G-VPF) technique outperforms the state of the art, not only in terms of double compression detection and GOP size estimation, but also in reducing computational time.

Video forensics keeps developing new technologies to verify the authenticity and the integrity of digital videos. While most of the existing methods rely on the analysis of the video data stream, recently, a new line of research was introduced to investigate video life cycle based on the analysis of the video container. Anyway, existing contributions in this field are based on manual comparison of video container structure and content, which is time demanding and error-prone. In this paper, we introduce a method for unsupervised analysis of video file containers, and present two main forensic applications of such method: the first one deals with video integrity verification, based on the dissimilarity between a reference and a query file container; the second one focuses on the identification and classification of the source device brand, based on the analysis of containers structure and content. Noticeably, the latter application relies on the likelihood-ratio framework, which is more and more approved by the forensic community as the appropriate way to exhibit findings in court. We tested and proved the effectiveness of both applications on a dataset composed by 578 videos taken with modern smartphones from major brands and models. The proposed approaches are proved to be valuable also for requiring an extremely small computational cost as opposed to all available techniques based on the video stream analysis or manual inspection of file containers.

Forensic research community keeps proposing new techniques to analyze digital images and videos. However, the performance of proposed tools are usually tested on data that are far from reality in terms of resolution, source device, and processing history. Remarkably, in the latest years, portable devices became the preferred means to capture images and videos, and contents are commonly shared through social media platforms (SMPs, for example, Facebook, YouTube, etc.). These facts pose new challenges to the forensic community: for example, most modern cameras feature digital stabilization, that is proved to severely hinder the performance of video source identification technologies; moreover, the strong re-compression enforced by SMPs during upload threatens the reliability of multimedia forensic tools. On the other hand, portable devices capture both images and videos with the same sensor, opening new forensic opportunities. The goal of this paper is to propose the VISION dataset as a contribution to the development of multimedia forensics. The VISION dataset is currently composed by 34,427 images and 1914 videos, both in the native format and in their social version (Facebook, YouTube, and WhatsApp are considered), from 35 portable devices of 11 major brands. VISION can be exploited as benchmark for the exhaustive evaluation of several image and video forensic tools.

In this paper, we propose two different solutions for making a recently proposed asymmetric fingerprinting protocol based on client-side embedding robust to collusion attacks. The first solution is based on projecting a client-owned random fingerprint, securely obtained through existing cryptographic protocols, using for each client a different random matrix generated by the server. The second solution consists in assigning to each client a Tardos code, which can be done using existing asymmetric protocols, and modulating such codes using a specially designed random matrix. Suitable accusation strategies are proposed for both solutions, and their performance under the averaging attack followed by the addition of Gaussian noise is analytically derived. Experimental results show that the analytical model accurately predicts the performance of a realistic system. Moreover, the results also show that the solution based on independent random projections outperforms the solution based on Tardos codes, for different choices of parameters and under different attack models.