As AI-generated synthetic images become increasingly realistic, Vision Transformers (ViTs) have emerged as a cornerstone of modern deepfake detection. However, the prevailing reliance on frozen, pre-trained backbones introduces a subtle yet critical vulnerability. In this work, we present the Surrogate Iterative Adversarial Attack (SIAA), a gray-box attack that exploits knowledge of the detector’s ViT backbone alone and operates entirely within the target detector’s feature space to craft highly effective adversarial examples. Through our experiments, involving multiple ViT-based detectors and diverse gray-box scenarios, including few-shot learning, complete training misalignment and attack transferability tests, we demonstrate that this vulnerability consistently yields high attack success rates, often approaching white-box performance. By doing so, we reveal that backbone knowledge alone is sufficient to undermine detector reliability, highlighting the urgent need for more resilient defenses in adversarial multimedia forensics.

The rapid evolution of generative artificial intelligence has made video inpainting and object removal highly realistic, posing a severe threat to multimedia integrity. While various forensic detectors have been proposed, they predominantly rely on high frequency noise or specific artefact signatures that are easily destroyed by real world degradations like H.264 and HEVC compression, and AI based post processing. To address this critical gap, we introduce AINPAINT, a large scale forensic dataset containing over 25,000 video sequences manipulated with nine diverse generative techniques, explicitly including variants subjected to temporal smoothing and heavy compression. On top of AINPAINT, we propose two complementary architectures for video inpainting localization built upon a LoRA adapted DINOv2 backbone. The first method extracts rich semantic spatial features, while the second augments these features with temporal motion anomalies derived from dense optical flow. Beyond merely establishing new performance baselines, our ablation provides a functional decision guide for the forensics community, clarifying when spatial features alone are preferable and when motion anomalies provide a measurable gain in the presence of post-processing, H.264 and HEVC compression and data-shifts. The dataset and code implementation are available at:

Recent advancements in AI-based multimedia generation have enabled the creation of hyper-realistic images and videos, raising concerns about their potential use in spreading misinformation. The widespread accessibility of generative techniques, which allow for the production of fake multimedia from prompts or existing media, along with their continuous refinement, underscores the urgent need for highly accurate and generalizable AI-generated media detection methods, underlined also by new regulations like the European Digital AI Act. In this paper, we draw inspiration from Vision Transformer (ViT)-based fake image detection and extend this idea to video. We propose an original framework that effectively integrates ViT embeddings over time to enhance detection performance. Our method shows promising accuracy, generalization, and few-shot learning capabilities across a new, large and diverse dataset of videos generated using five open source generative techniques from the state-of-the-art, as well as a separate dataset containing videos produced by proprietary generative methods.

Synthetic image source attribution is an open challenge, with an increasing number of image generators being released yearly. The complexity and the sheer number of available generative techniques, as well as the scarcity of high-quality open source datasets of diverse nature for this task, make training and benchmarking synthetic image source attribution models very challenging. WILD1is a new in-the-Wild Image Linkage Dataset designed to provide a powerful training and benchmarking tool for synthetic image attribution models. The dataset is built out of a closed set of 10 popular commercial generators, which constitutes the training base of attribution models, and an open set of 10 additional generators, simulating a real-world in-the-wild scenario. Each generator is represented by 1,000 images, for a total of 10,000 images in the closed set and 10,000 images in the open set. Half of the images are post-processed with a wide range of operators. WILD allows benchmarking attribution models in a wide range of tasks, including closed and open set identification and verification, and robust attribution with respect to post-processing and adversarial attacks. Models trained on WILD are expected to benefit from the challenging scenario represented by the dataset itself. Moreover, an assessment of seven baseline methodologies on closed and open set attribution is presented, including robustness tests with respect to post-processing.

AI-generated synthetic media are increasingly used in real-world scenarios, often with the purpose of spreading misinformation and propaganda through social media platforms, where compression and other processing can degrade fake detection cues. Currently, many forensic tools fail to account for these in-the-wild challenges. In this work, we introduce TrueFake, a large-scale benchmarking dataset of 600,000 images including top notch generative techniques and sharing via three different social networks. This dataset allows for rigorous evaluation of state-of-the-art fake image detectors under very realistic and challenging conditions. Through extensive experimentation, we analyze how social media sharing impacts detection performance, and identify current most effective detection and training strategies. Our findings highlight the need for evaluating forensic models in conditions that mirror real-world use.

Forensic image source attribution aims at deciding whether a query image was taken by a specific camera. While various algorithms leveraging forensic traces have been proposed, the most effective techniques rely on Photo Response Non-Uniformity (PRNU), a pattern introduced by camera sensors during the image acquisition process. In recent years, advances in image acquisition and processing technologies in modern devices have been found to impact the performance of PRNU, seemingly challenging its uniqueness. In this paper, we build upon recent discoveries of leaks in PRNU uniqueness, focusing on the dataset recently published by Iuliani et al. which has been instrumental in identifying numerous issues related to source attribution. Specifically, we analyze the effects in terms of false positive of visible watermarks applied to Xiaomi Mi 9 images, and reveal artifacts in the magnitude of the Discrete Fourier Transform of Samsung A50 images, indicative of the absence of non-unique artifacts. Furthermore, we demonstrate how several false positive cases are attributed to mislabeled devices. Finally, we show that a number of false negatives from the dataset are traceable to radially corrected images, and to images processed by third-party software that had not been previously noticed.

The term “mimicry attack” has been coined in computer security and used in adversarial machine learning: an attacker observes what a machine-learning system has learned and adjusts the malicious input so that it mimics a benign input. In this paper we extend this concept to image forensics, to allow an attacker modifying a manipulated image so that it appears pristine when analyzed by a target forensic detector. Recent work has shown that such attacks can be executed against detectors based on deep networks for hiding image tampering. We do more than that: our mimicry attack can force the target detector to identify arbitrary fictitious manipulations, while hiding the true ones. Accordingly, the user of the forensic detector is completely misled. From a methodological viewpoint, the proposed attack artificially alters the detector-specific intermediate representations according to the pixel distribution in the manipulated image, by applying a gradient-based optimization process. Experimental tests on different data sets and detectors demonstrate that our approach succeeds in jointly hiding manipulated areas and arbitrarily adding new ones, favorably comparing with the state-of-the-art in the first task.

The amount of multimedia content shared everyday, combined with the level of realism reached by recent fake-generating technologies, threatens to impair the trustworthiness of online information sources. The process of uploading and sharing data tends to hinder standard media forensic analyses, since multiple re-sharing steps progressively hide the traces of past manipulations. At the same time though, new traces are introduced by the platforms themselves, enabling the reconstruction of the sharing history of digital objects, with possible applications in information flow monitoring and source identification. In this work, we propose a supervised framework for the reconstruction of image sharing chains on social media platforms. The system is structured as a cascade of backtracking blocks, each of them tracing back one step of the sharing chain at a time. Blocks are designed as ensembles of classifiers trained to analyse the input image independently from one another by leveraging different feature representations that describe both content and container of the media object. Individual decisions are then properly combined by a late fusion strategy. Results highlight the advantages of employing multiple clues, which allow accurately tracing back up to three steps along the sharing chain.

With today's technologies, the possibility to generate highly realistic visual fakes is within everyone's reach, leading to major threats in terms of misinformation and data trustworthiness. This holds in particular for synthetically generated faces, which are able to deceive even the most experienced observers, and can be exploited to create fake digital identities with synthetic facial attributes, to be used on social networks and online services. In response to this threat, researchers have employed artificial intelligence to detect synthetic images by analysing patterns and artifacts introduced by the generative models. However, most online images are subject to repeated sharing operations by social media platforms. Said platforms process uploaded images by applying operations (like compression) that progressively degrade those useful forensic traces, compromising the effectiveness of the developed detectors. To solve the synthetic-vs-real problem “in the wild”, more realistic image databases are needed to train specialised detectors. In this work, we present TrueFace, a first dataset of social-media-processed real and synthetic faces, obtained by the successful StyleGAN generative models, and shared on Facebook, Twitter and Telegram. The dataset is used to validate a ResNet-based image classification model addressing the discrimination of synthetic-vs-real faces in both presocial and post-social scenarios. The results demonstrate that even detectors with extremely high performance on non-shared images struggle to retain their accuracy on images from social media, while fine-tuning with shared images strongly mitigates such performance issues.

Video stabilization is an in-camera processing commonly applied by modern acquisition devices. While significantly improving the visual quality of the resulting videos, it has been shown that such operation typically hinders the forensic analysis of video signals. In fact, the correct identification of the acquisition source usually based on Photo Response non-Uniformity (PRNU) is subject to the estimation of the transformation applied to each frame in the stabilization phase. A number of techniques have been proposed for dealing with this problem, which however typically suffer from a high computational burden due to the grid search in the space of inversion parameters. Our work attempts to alleviate these short-comings by exploiting the parallelization capabilities of Graphics Processing Units (GPUs), typically used for deep learning applications, in the framework of stabilised frames inversion. Moreover, we propose to exploit SIFT features to estimate the camera momentum and identify less stabilized temporal segments, thus enabling a more accurate identification analysis, and to efficiently initialize the frame-wise parameter search of consecutive frames. Experiments on a consolidated benchmark dataset confirm the effectiveness of the proposed approach in reducing the required computational time and improving the source identification accuracy. The code is available at https://github.com/AMontiB/GPU-PRNU-SIFT.

With social networks reaching unprecedented numbers of active users and data traffic worldwide, forensic scientists have been working to secure the trustworthiness of online information against the threat of misinformation. Several works have already investigated the possibility to trace malicious contents back to its perpetrators by following the chain of sharing operations. In real-world scenarios, however, users also have the chance to upload images in multiple ways, such as via smartphone apps or desktop browsers. Being able to detect different sharing modalities may represent a valuable and still unexplored insight for forensic purposes. Following this line, in this work we present SHADE, a first collection of real-world images shared from different devices, operating systems, and user interfaces. This database provides the forensic community with a new asset for investigating the peculiar artifacts introduced by different sharing modes, and for validating detection algorithms. The dataset was evaluated by applying a set of feature descriptors borrowed from platform provenance analysis, allowing us to reach promising results in the classification of sharing modalities.

Abstract The dependability of visual information on the web and the authenticity of digital media appearing virally in social media platforms has been raising unprecedented concerns. As a result, in the last years the multimedia forensics research community pursued the ambition to scale the forensic analysis to real-world web-based open systems. This survey aims at describing the work done so far on the analysis of shared data, covering three main aspects: forensics techniques performing source identification and integrity verification on media uploaded on social networks, platform provenance analysis allowing to identify sharing platforms, and multimedia verification algorithms assessing the credibility of media objects in relation to its associated textual information. The achieved results are highlighted together with current open issues and research challenges to be addressed in order to advance the field in the next future.

Social Networks (SN) and Instant Messaging Apps (IMA) are more and more engaging people in their personal relations taking possession of an important part of their daily life. Huge amounts of multimedia contents, mainly photos, are poured and successively shared on these networks so quickly that is not possible to follow their paths. This last issue surely grants anonymity and impunity thus it consequently makes easier to commit crimes such as reputation attack and cyberbullying. In fact, contents published within a restricted group of friends on an IMA can be rapidly delivered and viewed on a SN by acquaintances and then by strangers without any sort of tracking. In a forensic scenario (e.g., during an investigation), succeeding in understanding this flow could be strategic, thus allowing to reveal all the intermediate steps a certain content has followed. This work aims at tracking multiple sharing on social networks, by extracting specific traces left by each SN within the image file, due to the process each of them applies, to perform a multi-class classification. Innovative strategies, based on deep learning, are proposed and satisfactory results are achieved in recovering till triple up-downloads.

The majority of news published online presents one or more images or videos, which make the news more easily consumed and therefore more attractive to huge audiences. As a consequence, news with catchy multimedia content can be spread and get viral extremely quickly. Unfortunately, the availability and sophistication of photo editing software are erasing the line between pristine and manipulated content. Given that images have the power of bias and influence the opinion and behavior of readers, the need of automatic techniques to assess the authenticity of images is straightforward. This paper aims at detecting images published within online news that have either been maliciously modified or that do not represent accurately the event the news is mentioning. The proposed approach composes image forensic algorithms for detecting image tampering, and textual analysis as a verifier of images that are misaligned to textual content. Furthermore, textual analysis can be considered as a complementary source of information supporting image forensics techniques when they falsely detect or falsely ignore image tampering due to heavy image postprocessing. The devised method is tested on three datasets. The performance on the first two shows interesting results, with F1-score generally higher than 75%. The third dataset has an exploratory intent; in fact, although showing that the methodology is not ready for completely unsupervised scenarios, it is possible to investigate possible problems and controversial cases that might arise in real-world scenarios.

Clustering images according to their acquisition devices is a well-known problem in multimedia forensics, which is typically faced by means of camera sensor pattern noise (SPN). Such an issue is challenging since SPN is a noise-like signal, hard to be estimated, and easy to be attenuated or destroyed by many factors. Moreover, the high dimensionality of SPN hinders large-scale applications. Existing approaches are typically based on the correlation among SPNs in the pixel domain, which might not be able to capture intrinsic data structure in the union of vector subspaces. In this paper, we propose an accurate clustering framework, which exploits linear dependences among SPNs in their intrinsic vector subspaces. Such dependences are encoded under sparse representations, which are obtained by solving an LASSO problem with non-negativity constraint. The proposed framework is highly accurate in a number of clusters' estimation and image association. Moreover, our framework is scalable to the number of images and robust against double JPEG compression as well as the presence of outliers, owning big potential for real-world applications. Experimental results on Dresden and Vision database show that our proposed framework can adapt well to both medium-scale and large-scale contexts and outperforms the state-of-the-art methods.

Recognizing fake images in online news is a challenging problem. This is especially true in the case of critical situations, when journalists might insert high-impact images to make a piece of news more appealing to the readers, neglecting to check their authenticity and provenance. Given the importance of this task, in the literature, it is possible to find several attempts to solve the problem from different points of view. This paper faces the specific problem of recognizing images in online news which have been modified or mis-contextualized, i.e. images taken in a different place and/or time with respect to the event to which they are associated. To identify image tampering a number of image forensic techniques were exploited and combined. On the other hand, for mis-contextualization detection, a textual analysis approach is proposed based on the extraction of features from the news the image is associated with, and from textual information retrieved online using the image at stake as pivot. The obtained results are rather satisfactory on laboratory data, with results that in some cases improve the state of the art for image forensics. The method was tested on three datasets, one of which already used in the literature, while the others created ad-hoc to further investigate its performances.

Studying the impact of sharing platforms like social networks and messaging services on multimedia content nowadays represents a due step in multimedia forensics research. In this framework, we study the characteristics of images that are uploaded and shared through three popular mobile messaging apps combined with two different sending mobile operating systems (OS). In our analysis, we consider information contained both in the image signal and in the metadata of the image file. We show that it is generally possible to identify a posteriori the last app and the OS that have been used for uploading. This is done by considering different scenarios involving images shared both once and twice. Moreover, we show that, by leveraging the knowledge of the last sharing app and system, it is possible to retrieve information on the previous sharing step for double shared images. In relation to prior works, a discussion on the influence of the rescaling and recompression mechanism - usually performed differently through apps and OSs - is also proposed, and the feasibility of retrieving the compression parameters of the image before being shared is assessed.

This paper proposes a novel two-stage framework for event recognition in still images. First, for a generic event image, deep features, obtained via different pre-trained models, are fed into an ensemble of classifiers, whose posterior classification probabilities are thereafter fused by means of an order-induced scheme, which penalizes the yielded scores according to their confidence in classifying the image at hand, and then averages them. Second, we combine the fusion results with a reverse matching paradigm in order to draw the final output of our proposed pipeline. We evaluate our approach on three challenging datasets and we show that better results can be attained, advancing recent leading works.

Morphological operators are widely used in binary image processing for several purposes, such as removing noise, detecting contours or particular structures, and regularizing shapes. In particular, morphological filters are largely adopted in scanned documents to correct the artifacts caused by acquisition and binarization, as well as other processing. In this paper, we propose a novel approach for forensics detection of morphological filtering on binary images. The proposed technique exploits some mathematical properties of the two basic morphologic operators, erosion and dilation, to define an algorithm able not only to detect the application of the filter, but also to estimate the shape of the relevant structuring element. Experimental tests demonstrate that the technique is effective and robust to the most common operations performed on binary image documents.

Intrinsic statistical properties of natural uncompressed images are used in image forensics for detecting the traces of previous processing operations. In this paper, we propose novel forensic detectors of JPEG compression traces in images stored in uncompressed formats, based on a theoretical analysis of Benford-Fourier coefficients computed on the 8 × 8 block-Discrete Cosine Transform (DCT) domain. In fact, the distribution of such coefficients is derived theoretically both under the hypotheses of no compression and previous compression with a certain quality factor, allowing for the computation of the respective likelihood functions. Then, two classification tests based on different statistics are proposed, both relying on a discriminative threshold that can be determined without the need of any training phase. The statistical analysis is based on the only assumptions of generalized Gaussian distribution of DCT coefficients and independence among DCT frequencies, thus resulting in robust detectors applying to any uncompressed image. In fact, experiments on different datasets show that the proposed models are suitable for the images of different sizes and source cameras, thus overcoming dataset-dependence issues that typically affect the state-of-art techniques.

In this paper we address the issue of photo galleries synchronization, where pictures related to the same event are collected by different users. Existing solutions to address the problem are usually based on unrealistic assumptions, like time consistency across photo galleries, and often heavily rely on heuristics, therefore limiting the applicability to real-world scenarios. We propose a solution that achieves better generalization performance for the synchronization task compared to the available literature. The method is characterized by three stages: at first, deep convolutional neural network features are used to assess the visual similarity among the photos; then, pairs of similar photos are detected across different galleries and used to construct a graph; eventually, a probabilistic graphical model is used to estimate the temporal offset of each pair of galleries, by traversing the minimum spanning tree extracted from this graph. The experimental evaluation is conducted on four publicly available datasets covering different types of events, demonstrating the strength of our proposed method. A thorough discussion of the obtained results is provided for a critical assessment of the quality in synchronization.

In this article, we present a novel framework that can produce a visual description of a tourist attraction by choosing the most diverse pictures from community-contributed datasets, which describe different details of the queried location. The main strength of the proposed approach is its flexibility that permits us to filter out non-relevant images and to obtain a reliable set of diverse and relevant images by first clustering similar images according to their textual descriptions and their visual content and then extracting images from different clusters according to a measure of the user’s credibility. Clustering is based on a two-step process, where textual descriptions are used first and the clusters are then refined according to the visual features. The degree of diversification can be further increased by exploiting users’ judgments on the results produced by the proposed algorithm through a novel approach, where users not only provide a relevance feedback but also a diversity feedback. Experimental results performed on the MediaEval 2015 “Retrieving Diverse Social Images” dataset show that the proposed framework can achieve very good performance both in the case of automatic retrieval of diverse images and in the case of the exploitation of the users’ feedback. The effectiveness of the proposed approach has been also confirmed by a small case study involving a number of real users.

In this paper, we propose a novel approach for face spoofing detection using the high-order Local Derivative Pattern from Three Orthogonal Planes (LDP-TOP). The proposed method is not only simple to derive and implement, but also highly efficient, since it takes into account both spatial and temporal information in different directions of subtle face movements. According to experimental results, the proposed approach outperforms state-of-the-art methods on three reference datasets, namely Idiap REPLAY-ATTACK, CASIA-FASD, and MSU MFSD. Moreover, it requires only 25 video frames from each video, i.e., only one second, and thus potentially can be performed in real time even on low-cost devices.

We address the known problem of detecting a previous compression in JPEG images, focusing on the challenging case of high and very high quality factors (>= 90) as well as repeated compression with identical or nearly identical quality factors. We first revisit the approaches based on Benford--Fourier analysis in the DCT domain and block convergence analysis in the spatial domain. Both were originally conceived for specific scenarios. Leveraging decision tree theory, we design a combined approach complementing the discriminatory capabilities. We obtain a set of novel detectors targeted to high quality grayscale JPEG images.

Event discovery from single pictures is a challenging problem that has raised significant interest in the last decade. During this time, a number of interesting solutions have been proposed to tackle event discovery in still images. However, a large scale benchmarking image dataset for the evaluation and comparison of event discovery algorithms from single images is still lagging behind. To this aim, in this paper we provide a large-scale properly annotated and balanced dataset of 490,000 images, covering every aspect of 14 different types of social events, selected among the most shared ones in the social network. Such a large scale collection of event-related images is intended to become a powerful support tool for the research community in multimedia analysis by providing a common benchmark for training, testing, validation and comparison of existing and novel algorithms. In this paper, we provide a detailed description of how the dataset is collected, organized and how it can be beneficial for the researchers in the multimedia analysis domain. Moreover, a deep learning based approach is introduced into event discovery from single images as one of the possible applications of this dataset with a belief that deep learning can prove to be a breakthrough also in this research area. By providing this dataset, we hope to gather research community in the multimedia and signal processing domains to advance this application.

Research on event-based processing and analysis of media is receiving an increasing attention from the scientific community due to its relevance for an abundance of applications, from consumer video management and video surveillance to lifelogging and social media.Events have the ability to semantically encode relationships of different informational modalities, such as visual-audio-text, time, involved agents and objects, with the spatio-temporal component of events being a key feature for contextual analysis.This unveils an enormous potential for exploiting new information sources and opening new research directions.In this paper, we survey the existing literature in this field.We extensively review the employed conceptualization of the notion of event in multimedia, the techniques for event representation and modeling, the feature representation and event inference approaches for the problems of event detection in audio, visual, and textual content.Furthermore, we review some key event-based multimedia applications, and various benchmarking activities that provide solid frameworks for measuring the performance of different event processing and analysis systems.We provide an in-depth discussion of the insights obtained from reviewing the literature and identify future directions and challenges.

In this paper, we propose a forensic technique that is able to detect the application of a median filter to 1D data. The method relies on deterministic mathematical properties of the median filter, which lead to the identification of specific relationships among the sample values that cannot be found in the filtered sequences. Hence, their presence in the analyzed 1D sequence allows excluding the application of the median filter. Owing to its deterministic nature, the method ensures 0% false negatives, and although false positives (sequences not filtered classified as filtered) are theoretically possible, experimental results show that the false alarm rate is null for sufficiently long sequences. Furthermore, the proposed technique has the capability to locate with good precision a median filtered part of 1-D data and provides a good estimate of the window size used.

In this paper, we present a novel method that can produce a visual description of a landmark by choosing the most diverse pictures that best describe all the details of the queried location from community-contributed datasets. The main idea of this method is to filter out non-relevant images at a first stage and then cluster the images according to textual descriptors first, and then to visual descriptors. The extraction of images from different clusters according to a measure of user's credibility, allows obtaining a reliable set of diverse and relevant images. Experimental results performed on the MediaEval 2014 “Retrieving Diverse Social Images” dataset show that the proposed approach can achieve very good performance outperforming state-of-art techniques.

Digital forensics is a relatively new research area which aims at authenticating digital media by detecting possible digital forgeries. Indeed, the ever increasing availability of multimedia data on the web, coupled with the great advances reached by computer graphical tools, makes the modification of an image and the creation of visually compelling forgeries an easy task for any user. This in turns creates the need of reliable tools to validate the trustworthiness of the represented information. In such a context, we present here RAISE, a large dataset of 8156 high-resolution raw images, depicting various subjects and scenarios, properly annotated and available together with accompanying metadata. Such a wide collection of untouched and diverse data is intended to become a powerful resource for, but not limited to, forensic researchers by providing a common benchmark for a fair comparison, testing and evaluation of existing and next generation forensic algorithms. In this paper we describe how RAISE has been collected and organized, discuss how digital image forensics and many other multimedia research areas may benefit of this new publicly available benchmark dataset and test a very recent forensic technique for JPEG compression detection.

Modern computer graphics technologies brought realism in computer-generated characters, making them achieve truly natural appearance. Besides traditional virtual reality applications such as avatars, games, or cinema, these synthetic characters may be used to generate realistic fakes, which may lead to improper use of the technology. This fact raises the demand for advanced tools able to discriminate real and artificial human faces in digital media. In this paper, we propose a method to distinguish between computer generated and natural faces by modeling and evaluating their dynamic behavior. Because of a 3D-model-based video analysis, the proposed technique allows identifying synthetic characters by detecting their more limited variability over time. Experimental results demonstrate the effectiveness of the proposed approach also on very challenging and realistic video sequences.

The concept of “event” emerged in recent years as a key feature to efficiently index and retrieve media. Several approaches have been proposed to analyze the relationship between events and media, enable event discovery, and perform event-based media tagging, indexing, and retrieval. Despite the outstanding work done in this area, a major problem that remains open is how to infer the link between visual concepts and events. In particular, the possibility of understanding which perceptual elements allow a human recognizing the event depicted by an image would open new directions in event media discovery. In this paper we introduce the concept of event saliency to define the above event-revealing perceptual elements, and we propose an original method to detect it by exploiting crowd knowledge through gamification. We propose an adversarial game with a hidden purpose, where users are engaged in competitive roles: masking photos to prevent competitors recognizing the related event, and discovering events in photos masked by other players. Rules and incentives are defined to minimize cheating and force players to focus on details that really matter. A suitable algorithm composes the masks created by different players on the same media, thus producing a saliency map that, different from the traditional concept of saliency, does not focus on perceptual prominence but rather on event-related semantics of media. A thorough validation on public datasets is presented, and initial experiments to apply event saliency in detection tasks are proposed . Furthermore, an event-saliency dataset is disclosed to allow further research.

First-order statistics of First Significant Digits (FSD) have been recently exploited in multimedia forensics as a powerful tool to reveal traces of previous coding operations. As an answer, adversarial approaches aimed at modifying the FSD histogram and fooling such forensic methods have been proposed. However, the existing techniques have limitations in terms of distortion introduced in the multimedia object. In this paper, a transportation-theoretic formulation of the problem is presented which provides a close-to-optimal solution. Such strategy is tested in a well-known image forensic scenario, where FSDs of 8 × 8-DCT coefficients after single or double quantization are modified in order to restore a certain target histogram and the distortion with respect to the provided compressed image is measured in terms of MSE.

We describe a new forensic technique for distinguishing between computer generated and human faces in video. This technique identifies tiny fluctuations in the appearance of a face that result from changes in blood flow. Because these changes result from the human pulse, they are unlikely to be found in computer generated imagery. We use the absence or presence of this physiological signal to distinguish computer generated from human faces.

Given the recent development of advanced multimedia techniques able to support the creation of realistic computer generated characters, there is the parallel need of automatic tools allowing users to verify the source of the multimedia data they are observing, thus discriminating between artificial and natural information. In this paper, we focus on video representing human beings and we propose a novel method to identify computer generated characters by analysing the evolution of the face model in chronological order. Experimental results show that photorealistic facial animations, which are usually performed following fixed patterns, can be distinguished from natural ones, which follow much more complicated and various geometric distortions.

Intrinsic statistical properties of natural uncompressed images can be used in image forensics for detecting traces of previous processing operations. In this paper, we extend the recent theoretical analysis of Benford-Fourier coefficients and propose a novel forensic detector of JPEG compression traces in images stored in an uncompressed format. The classification is based on a binary hypothesis test for which we can derive theoretically the confidence intervals, thus avoiding any training phase. Experiments on real images and comparisons with state-of-art techniques show that the proposed detector outperforms existing ones and overcomes issues due to dataset-dependency.

In this contribution, we propose a watershed-based method with support from external data sources and visual information to detect social events in web multimedia. The idea is based on two main observations: (1) people cannot be involved in more than one event at the same time, and (2) people tend to introduce similar annotations for all images associated to the same event. Based on these observations, the metadata is turned to an image so that each row contains all records belonging to one user; and these records are sorted by time. Thus, the social event detection is turned to watershed-based image segmentation, where Markers are generated by using (keyword, location, visual) features with support of external data sources, and the Flood progress is carried on by taking into account (tags set, time, visual) features. We test our algorithm on the MediaEval 2012 dataset both using only external data but also introducing visual information.

Median filtering is a well-known non linear denoising filter often used as an harmless post-processing, sometimes also employed to affect the reliability of some forensic techniques. In this work, we present a novel counter-forensic method able to conceal the characteristic traces left by median filtering. By exploiting the knowledge of features used in existing median filtering detectors, we are able to remove the characteristic footprints via suitable random pixel modification, while keeping the quality of the counter-attacked image high. Experimental results show that the proposed method is very effective, computationally efficient and competitive with other state-of-the-art techniques.

In this paper we present a method of visual salient region detection based on depth maps estimated from 2D images. Depth estimation aims at better understanding spatial scene layout and relationship between objects. From depth maps we extract geometry related features that are further fused with color contrast. We solve saliency detection problem in segment-wise domain that allows prediction of objects rather than separate pixels. Modelling of saliency is done using conditional random field that allows for pairwise dependencies of segments. Parameters tuning is done by learning from ground-truth data. The evaluation has shown feasibility and good performance of the proposed method.

Traces left by lossy compression processes have been widely studied in digital image forensics. In particular, the artifacts produced by JPEG compression have been characterized and exploited both in forensic methods and counter-forensic attacks. In this paper, we propose a novel anti-forensic procedure, aimed at concealing the traces of single JPEG compression by recovering the original distribution of first significant digits (FSD) of the DCT coefficients. We analyze the performance of our method and compare it with anti-forensic attacks reported in the literature in terms of quality of the resulting image. In addition, we prove the effectiveness of our approach as counter-forensic processing by measuring its impact on the performance of two different forensic tools, applied after the anti-forensic action.

The large diffusion of photo cameras makes quite common that an event is acquired from different devices, conveying different subjects and perspectives of the same happening. Often, these photo collections are shared among different users through social networks and networked communities. Automatic tools are more and more used to support the users in organizing such archives, and it is largely accepted that time/space information is fundamental to this purpose. Unfortunately, both data are often unreliable, and in particular, timestamps may be affected by erroneous or imprecise setting of the camera clock, thus making the retrieval based on temporal tagging unreliable. In this paper, we propose to solve this well-known problem by introducing a synchronization algorithm that exploit the content of pictures to estimate the mutual delays among different cameras, thus achieving an a-posteriori synchronization of various photo collections referring to the same event. Experimental results show that, for sufficiently large archives, a notable accuracy can be achieved in the estimation of the synchronization information.

Events are becoming very popular as a tool to organize and access large media collections. An unsolved problem however, is how to define event models. Most part of the approaches so far proposed in the literature are based on a-priori knowledge, and translate into hierarchical data structures or taxonomies a more or less intuitive definition of what a given type of event is. The association of media and event models is then a consequent process, in which one tries to learn the distinctive characteristics of media associated to a certain event or sub-event. In this paper, we attempt to reverse this paradigm, inferring from a set of media collections belonging to the same event class the underlying taxonomy in an unconstrained way. As a result we obtain a hierarchy of natural clusters, largely shared by the different collections, which capture the essence of the event itself. Although it is not possible to compare the proposed approach with state-of-the-art method based on a-priori event structures, experimental results demonstrate that this approach may become an effective support for discovering and defining event models and managing event-related data collections.

This paper presents a novel method of visual saliency detection. The use of saliency promises benefits to multimedia applications. However, up to now just few reasonable applications of saliency exist. It is clear that limited accuracy is one of the possible reasons for this. Another reason could be that in general saliency allows us to detect salient regions of the image rather than objects. To fill this gap we study to what extend the integration of segmentation into saliency detection allows the estimation of saliency of objects. In this paper we propose a method that operates with segments rather than with separate pixels. The comparison with state-of-the-art methods shows that our method is successful in highlighting the mass of the object of interest. Finally, we discuss possible directions for the further work.

In this paper we present a new near lossless reversible watermarking algorithm using adaptive prediction for embedding. The prediction is based on directional first-order differences of pixel intensities within a suitably selected neighborhood. The proposed scheme results to be computationally efficient and allows achieving high embedding capacity while preserving a high image quality. Extensive experimental results demonstrate the effectiveness of the proposed approach.

The manipulation of text on a sign or billboard is relatively easy to do in a way that is perceptually convincing. When text is on a planar surface and imaged under perspective projection, the text undergoes a specific distortion. When text is manipulated, it is unlikely to precisely satisfy this geometric mapping. We describe a technique for detecting if text in an image obeys the expected perspective projection, deviations from which are used as evidence of tampering.

Images are powerful means of communication. Adding images to documents, websites, magazines, helps attracting the attention and providing an immediate feeling about the content of the document itself. At the same time, images can be used to influence the attitude of the reader. In the digital era it is easier than ever to create and share multimedia documents making extensive use of visual data. Furthermore, it is extremely easy to modify and adapt the images in order to make them more suitable to convey a concept. These modifications may simply consist in focusing the attention on a specific part of the image, to heavier manipulations such as changing the visual appearance or the contents to bias the opinion of the observer. This creates a rising interest for the availability of automatic blind tools able to detect possible modifications of images, and to correlate these modifications with the relevant impact on the viewer. This paper proposes an example of application of these concepts that exploits a tool for automatic detection of contrast modifications and analyzes their impact on human feeling. The results of instrumental and subjective studies are presented and discussed.

In this paper we present the application of a recently developed benchmarking tool to the comparison of watermarking algorithms. We carry out an extensive analysis to assess and compare robustness of digital image watermarking techniques by considering the perceptual quality of un-marked images in terms of Weighted PSNR. Such a benchmarking tool employs genetic algorithms, introduces a novel metric for robustness assessment based on perceptual quality measures and allows analysis and comparison of different techniques performances. Experimental results show the effectiveness of the proposed approach.

In this work we present a framework for physical rehabilitation, which is based on hand tracking. One particular requirement in physical rehabilitation is the capability of the patient to correctly reproduce a specific path, following an example provided by the medical staff. Currently, these assignments are typically performed manually, and a nurse or doctor, who supervises the correctness of the movement, constantly assists the patient throughout the whole rehabilitation process. With the proposed system, our aim is to provide medical institutions and patients with a low-cost and portable instrument to automatically assess the rehabilitation improvements. To evaluate the performance of the exercise, and to determine the distance between the trial and the reference path, we adopted the dynamic time warping (DTW) and the longest common sub-sequence (LCSS) as discriminating metrics. Trajectories and numerical values are then stored to track the history of the patient and appraise the improvements of the rehabilitation process over time. Thanks to the tests conducted with real patients, it has been possible to evaluate the quality of the proposed tool, in terms of both graphical interface and functionalities.

This paper presents a novel and flexible benchmarking tool based on genetic algorithms (GA) and designed to assess the robustness of any digital image watermarking system. The main idea is to evaluate robustness in terms of perceptual quality, measured by weighted peak signal-to-noise ratio. Through a stochastic approach, we optimize this quality metric, by finding the minimal degradation that needs to be introduced in a marked image in order to remove the embedded watermark. Given a set of attacks, chosen according to the considered application scenario, GA support the optimization of the parameters to be assigned to each processing operation, in order to obtain an unmarked image with perceptual quality as high as possible. Extensive experimental results demonstrate the effectiveness of the proposed evaluation tool.

The present paper proposes an asymmetric watermarking scheme suitable for fingerprinting and precision-critical applications. The method is based on linear algebra and is proved to be secure under projection attack. The problem of anonymous fingerprinting is also addressed, by allowing a client to get a watermarked image from a server without revealing her own identity. In particular, we consider the specific scenario where the client is a structured organization being trusted as a whole but involving possibly untrusted members. In such a context, where the watermarked copy can be made available to all members, but only authorized subgroups should be able to remove the watermark and recover a distortion-free image, a multilevel access to the embedding key is provided by applying Birkhoff polynomial interpolation. Extensive simulations demonstrate the robustness of the proposed method against standard image degradation operators.

We introduce a novel asymmetric watermarking scheme, involving a private key for embedding and a public key for detection, and we detail its statistical analysis, relying on Neyman-Pearson criterion. The proposed scheme solves part of the problems connected to previous watermarking approaches based on linear algebra. In particular, special attention is paid at reducing the side information required at the detector, as well as at achieving higher robustness by enphasizing the contribution of the watermark in the detection phase.

The possibility of adding several watermarks to the same image would enable many interesting applications such as multimedia document tracing, data usage monitoring, multiple property management. In this paper, we present a novel watermarking scheme which allows to insert and reliably detect multiple watermarks sequentially embedded into a digital image. The proposed method, based on elementary linear algebra, is asymmetric, secure under projection attack and robust against distortion due to basic operations such as storage, transmission, format conversion, etc

The possibility of adding several watermarks to the same image would enable many interesting applications such as multimedia document tracing, data usage monitoring, multiple property management. In this paper, we present a novel watermarking scheme which allows to insert and reliably detect multiple watermarks sequentially embedded into a digital image, while maintaining a good perceptual quality. The proposed method, based on elementary linear algebra, is asymmetric, secure under projection attack and robust against noise addition and JPEG compression.

This paper presents a novel method for steganographic image watermarking with two main innovative features: i) it involves a hierarchical control, committing the watermark reconstruction to a trusted layered authority; and ii) it is deterministic, embedding a short meaningful signature into the cover image. Experimental results show that the embedded signature can be accurately recovered even in presence of a reasonable amount of image degradation due to image processing operators, such as filtering, geometric distorsions and compression.