The rapid evolution of generative artificial intelligence has made video inpainting and object removal highly realistic, posing a severe threat to multimedia integrity. While various forensic detectors have been proposed, they predominantly rely on high frequency noise or specific artefact signatures that are easily destroyed by real world degradations like H.264 and HEVC compression, and AI based post processing. To address this critical gap, we introduce AINPAINT, a large scale forensic dataset containing over 25,000 video sequences manipulated with nine diverse generative techniques, explicitly including variants subjected to temporal smoothing and heavy compression. On top of AINPAINT, we propose two complementary architectures for video inpainting localization built upon a LoRA adapted DINOv2 backbone. The first method extracts rich semantic spatial features, while the second augments these features with temporal motion anomalies derived from dense optical flow. Beyond merely establishing new performance baselines, our ablation provides a functional decision guide for the forensics community, clarifying when spatial features alone are preferable and when motion anomalies provide a measurable gain in the presence of post-processing, H.264 and HEVC compression and data-shifts. The dataset and code implementation are available at:

Current advances in human head modeling allow to generate plausible-looking 3D head models via neural representations, such as NeRFs and SDFs. Nevertheless, constructing complete high-fidelity head models with explicitly controlled animation remains an issue. Furthermore, completing the head geometry based on a partial observation, e.g. coming from a depth sensor, while preserving a high level of detail is often problematic for the existing methods. We introduce a generative model for detailed 3D head meshes on top of an articulated 3DMM which allows explicit animation and high-detail preservation at the same time. Our method is trained in two stages. First, we register a parametric head model with vertex displacements to each mesh of the recently introduced NPHM dataset of accurate 3D head scans. The estimated displacements are baked into a hand-crafted UV layout. Second, we train a StyleGAN model in order to generalize over the UV maps of displacements, which we later refer to HeadCraft. The decomposition of the parametric model and high-quality vertex displacements allows us to animate the model and modify the regions semantically. We demonstrate the results of unconditional sampling, fitting to a scan and editing. The code and data are available at https://seva100.github.io/headcraft.

Successful forensic detectors can produce excellent results in supervised learning benchmarks but struggle to transfer to real-world applications. We believe this limitation is largely due to inadequate training data quality. While most research focuses on developing new algorithms, less attention is given to training data selection, despite evidence that performance can be strongly impacted by spurious correlations such as content, format, or resolution. A well-designed forensic detector should detect generator specific artifacts rather than reflect data biases. To this end, we propose B-Free, a bias-free training paradigm, where fake images are generated from real ones using the conditioning procedure of stable diffusion models. This ensures semantic alignment between real and fake images, allowing any differences to stem solely from the subtle artifacts introduced by AI generation. Through content-based augmentation, we show significant improvements in both generalization and robustness over state-of-the-art detectors and more calibrated results across 27 different generative models, including recent releases, like FLUX and Stable Diffusion 3.5. Our findings emphasize the importance of a careful dataset design, highlighting the need for further research on this topic. Code and data are publicly available at https://grip-unina.github.io/B-Free/.

The rapid advancement of generative AI has revolutionized image creation, enabling high-quality synthesis from text prompts while raising critical challenges for media authenticity. We present AI-GenBench, a novel benchmark designed to address the urgent need for robust detection of AI-generated images in real-world scenarios. Unlike existing solutions that evaluate models on static datasets, AI-GenBench introduces a temporal evaluation framework where detection methods are incrementally trained on synthetic images, historically ordered by their generative models, to test their ability to generalize to new generative models, such as the transition from GANs to diffusion models. Our benchmark focuses on high-quality, diverse visual content and overcomes key limitations of current approaches, including arbitrary dataset splits, unfair comparisons, and excessive computational demands. AI-GenBench provides a comprehensive dataset, a standardized evaluation protocol, and accessible tools for both researchers and non-experts (e.g., journalists, fact-checkers), ensuring reproducibility while maintaining practical training requirements. By establishing clear evaluation rules and controlled augmentation strategies, AI-GenBench enables meaningful comparison of detection methods and scalable solutions. Code and data are publicly available to ensure reproducibility and to support the development of robust forensic detectors to keep pace with the rise of new synthetic generators1

Synthetic video generation is progressing very rapidly. The latest models can produce very realistic high-resolution videos that are virtually indistinguishable from real ones. Although several video forensic detectors have been recently proposed, they often exhibit poor generalization, which limits their applicability in a real-world scenario. Our key insight to overcome this issue is to guide the detector towards _seeing_ _what_ _really_ _matters_. In fact, a well-designed forensic classifier should focus on identifying intrinsic low-level artifacts introduced by a generative architecture rather than relying on high-level semantic flaws that characterize a specific model. In this work, first, we study different generative architectures, searching and identifying discriminative features that are unbiased, robust to impairments, and shared across models. Then, we introduce a novel forensic-oriented data augmentation strategy based on the wavelet decomposition and replace specific frequency-related bands to drive the model to exploit more relevant forensic cues. Our novel training paradigm improves the generalizability of AI-generated video detectors, without the need for complex algorithms and large datasets that include multiple synthetic generators. To evaluate our approach, we train the detector using data from a single generative model and test it against videos produced by a wide range of other models. Despite its simplicity, our method achieves a significant accuracy improvement over state-of-the-art detectors and obtains excellent results even on very recent generative models, such as NOVA and FLUX.

The ability to detect manipulated visual content is becoming increasingly important in many application fields, given the rapid advances in image synthesis methods. Of particular concern is the possibility of modifying the content of medical images, altering the resulting diagnoses. Despite its relevance, this issue has received limited attention from the research community. One reason is the lack of large and curated datasets to use for development and benchmarking purposes. Here, we investigate this issue and propose M3Dsynth, a large dataset of manipulated Computed Tomography (CT) lung images. We create manipulated images by injecting or removing lung cancer nodules in real CT scans, using three different methods based on Generative Adversarial Networks (GAN) or Diffusion Models (DM), for a total of 8,577 manipulated samples. Experiments show that these images easily fool automated diagnostic tools. We also tested several state-of-the-art forensic detectors and demonstrated that, once trained on the proposed dataset, they are able to accurately detect and localize manipulated synthetic content, even when training and test sets are not aligned, showing good generalization ability. Dataset and code are publicly available at https://grip-unina.github.io/M3Dsynth/.

Generalization is a main issue for current audio deepfake detectors, which struggle to provide reliable results on out-of-distribution data. Given the speed at which more and more accurate synthesis methods are developed, it is important to design techniques that work well also on data they were not trained for. In this paper we study the potential of large-scale pre-trained models for audio deepfake detection. To this end, the detection problem is reformulated as a speaker verification framework and fake audios are exposed by the mismatch between the voice sample under test and the voice of the claimed identity. With this paradigm, no fake speech sample is necessary in training, cutting off any link with the generation method at the root. Features are extracted by general-purpose large pre-trained models, with no need for training or fine-tuning on specific datasets. At detection time only a limited set of voice fragments of the identity under test is required. Experiments on several datasets show that detectors based on pre-trained models achieve excellent performance and show strong generalization ability, rivaling supervised methods on in-distribution data and largely overcoming them on out-of-distribution data.

High-quality artificially generated images are widely available now and increasingly realistic, posing challenges for image forensics in distinguishing them from real ones. Unfortunately, building a single detector that generalizes well to unseen generators is very difficult, creating the need for diverse cues. In this paper, we show that natural and synthetic images differ in their color statistics, possibly due to the widely used perceptual loss, which is more sensitive to brightness than to chroma differences. Consequently, color statistics offer valuable cues for forensic analysis and the development of robust detectors. Our experiments using simple hand-crafted color functions with a random forest achieve 91% accuracy averaged over all tested Diffusion Models, even with limited training samples.

In this paper we present TruFor, a forensic framework that can be applied to a large variety of image manipulation methods, from classic cheapfakes to more recent manipulations based on deep learning. We rely on the extraction of both high-level and low-level traces through a transformer-based fusion architecture that combines the RGB image and a learned noise-sensitive fingerprint. The latter learns to embed the artifacts related to the camera internal and external processing by training only on real data in a self-supervised manner. Forgeries are detected as deviations from the expected regular pattern that characterizes each pristine image. Looking for anomalies makes the approach able to robustly detect a variety of local manipulations, ensuring generalization. In addition to a pixel-level localization map and a whole-image integrity score, our approach outputs a reliability map that highlights areas where localization predictions may be error-prone. This is particularly important in forensic applications in order to reduce false alarms and allow for a large scale analysis. Extensive experiments on several datasets show that our method is able to reliably detect and localize both cheapfakes and deepfakes manipulations outperforming state-of-the-art works. Code is publicly available at https://grip-unina.github.io/TruFor/

Over the past decade, there has been tremendous progress in creating synthetic media, mainly thanks to the development of powerful methods based on generative adversarial networks (GAN). Very recently, methods based on diffusion models (DM) have been gaining the spotlight. In addition to providing an impressive level of photorealism, they enable the creation of text-based visual content, opening up new and exciting opportunities in many different application fields, from arts to video games. On the other hand, this property is an additional asset in the hands of malicious users, who can generate and distribute fake media perfectly adapted to their attacks, posing new challenges to the media forensic community. With this work, we seek to understand how difficult it is to distinguish synthetic images generated by diffusion models from pristine ones and whether current state-of-the-art detectors are suitable for the task. To this end, first we expose the forensics traces left by diffusion models, then study how current detectors, developed for GAN-generated images, perform on these new synthetic images, especially in challenging social-network scenarios involving image compression and resizing. Datasets and code are available at https:github.com/grip-unina/DMimageDetection.

State-of-the-art face recognition systems require vast amounts of labeled training data. Given the priority of privacy in face recognition applications, the data is limited to celebrity web crawls, which have issues such as limited numbers of identities. On the other hand, self-supervised revolution in the industry motivates research on the adaptation of related techniques to facial recognition. One of the most popular practical tricks is to augment the dataset by the samples drawn from generative models while preserving the identity. We show that a simple approach based on fine-tuning pSp encoder for StyleGAN allows to improve upon the state-of-the-art facial recognition and performs better compared to training on synthetic face identities. We also collect large-scale unlabeled datasets with controllable ethnic constitution – AfricanFaceSet-5M (5 million images of different people) and AsianFaceSet-3M (3 million images of different people) – and we show that pretraining on each of them improves recognition of the respective ethnicities (as well as others), while combining all unlabeled datasets results in the biggest performance increase. Our self-supervised strategy is the most useful with limited amounts of labeled training data, which can be beneficial for more tailored face recognition tasks and when facing privacy concerns. Evaluation is based on a standard RFW dataset and a new large-scale RB-WebFace benchmark. The code and data are made publicly available at https://github.com/seva100/stylegan-for-facerec.

In recent years there have been astonishing advances in AI-based synthetic media generation. Thanks to deep learning-based approaches it is now possible to generate data with a high level of realism. While this opens up new opportunities for the entertainment industry, it simultaneously undermines the reliability of multimedia content and supports the spread of false or manipulated information on the Internet. This is especially true for human faces, allowing to easily create new identities or change only some specific attributes of a real face in a video, so-called deepfakes. In this context, it is important to develop automated tools to detect manipulated media in a reliable and timely manner. This talk will describe the most reliable deep learning-based approaches for detecting deepfakes, with a focus on those that enable domain generalization [1]. The results will be presented on challenging datasets [2,3] with reference to realistic scenarios, such as the dissemination of manipulated images and videos on social networks. Finally, new possible directions will be outlined.

A major challenge in DeepFake forgery detection is that state-of-the-art algorithms are mostly trained to detect a specific fake method. As a result, these approaches show poor generalization across different types of facial manipulations, e.g., from face swapping to facial reenactment. To this end, we introduce ID-Reveal, a new approach that learns temporal facial features, specific of how a person moves while talking, by means of metric learning coupled with an adversarial training strategy. The advantage is that we do not need any training data of fakes, but only train on real videos. Moreover, we utilize high-level semantic features, which enables robustness to widespread and disruptive forms of post-processing. We perform a thorough experimental analysis on several publicly available benchmarks. Compared to state of the art, our method improves generalization and is more robust to low-quality videos, that are usually spread over social networks. In particular, we obtain an average improvement of more than 15% in terms of accuracy for facial reenactment on high compressed videos.

The advent of deep learning has brought a significant improvement in the quality of generated media. However, with the increased level of photorealism, synthetic media are becoming hardly distinguishable from real ones, raising serious concerns about the spread of fake or manipulated information over the Internet. In this context, it is important to develop automated tools to reliably and timely detect synthetic media. In this work, we analyze the state-of-the-art methods for the detection of synthetic images, highlighting the key ingredients of the most successful approaches, and comparing their performance over existing generative architectures. We will devote special attention to realistic and challenging scenarios, like media uploaded on social networks or generated by new and unseen architectures, analyzing the impact of suitable augmentation and training strategies on the detectors’ generalization ability.

The rise of deep learning has impacted profoundly all aspects of image processing and remote sensing. Following this trend, in the last few years, a large number of data-driven methods have been proposed also for SAR image despeckling. However, in spite of this large effort, only limited performance gains have been observed. We believe this is mostly due to the use of training sets that are only partially fit to the task, and sometimes plain wrong. In this work we assess experimentally the impact of training set design on the performance of SAR image despeckling with the goal of highlighting solid guidelines for sensible training.

The ever higher quality and wide diffusion of fake images have spawn a quest for reliable forensic tools. Many GAN image detectors have been proposed, recently. In real world scenarios, however, most of them show limited robustness and generalization ability. Moreover, they often rely on side information not available at test time, that is, they are not universal. We investigate these problems and propose a new GAN image detector based on a limited sub-sampling architecture and a suitable contrastive learning paradigm. Experiments carried out in challenging conditions prove the proposed method to be a first step towards universal GAN image detection, ensuring also good robustness to common image impairments, and good generalization to unseen architectures.

Lung ultrasound imaging is reaching growing interest from the scientific community. On one side, thanks to its harmlessness and high descriptive power, this kind of diagnostic imaging has been largely adopted in sensitive applications, like the diagnosis and follow-up of preterm newborns in neonatal intensive care units. On the other side, state-of-the-art image analysis and pattern recognition approaches have recently proven their ability to fully exploit the rich information contained in these data, making them attractive for the research community. In this work, we present a thorough analysis of recent deep learning networks and training strategies carried out on a vast and challenging multicenter dataset comprising 87 patients with different diseases and gestational ages. These approaches are employed to assess the lung respiratory status from ultrasound images and are evaluated against a reference marker. The conducted analysis sheds some light on this problem by showing the critical points that can mislead the training procedure and proposes some adaptations to the specific data and task. The achieved results sensibly outperform those obtained by a previous work, which is based on textural features, and narrow the gap with the visual score predicted by the human experts.

Abstract PRNU-based image processing is a key asset in digital multimedia forensics. It allows for reliable device identification and effective detection and localization of image forgeries, in very general conditions. However, performance impairs significantly in challenging conditions involving low quality and quantity of data. These include working on compressed and cropped images or estimating the camera PRNU pattern based on only a few images. To boost the performance of PRNU-based analyses in such conditions, we propose to leverage the image noiseprint, a recently proposed camera-model fingerprint that has proved effective for several forensic tasks. Numerical experiments on datasets widely used for source identification prove that the proposed method ensures a significant performance improvement in a wide range of challenging situations.

Source identification is an important topic in image forensics, since it allows to trace back the origin of an image. This represents a precious information to claim intellectual property but also to reveal the authors of illicit materials. In this letter we address the problem of device identification based on sensor noise and propose a fast and accurate solution using convolutional neural networks (CNNs). Specifically, we propose a 2-channel-based CNN that learns a way of comparing camera fingerprint and image noise at patch level. The proposed solution turns out to be much faster than the conventional approach and to ensure an increased accuracy. This makes the approach particularly suitable in scenarios where large databases of images are analyzed, like over social networks. In this vein, since images uploaded on social media usually undergo at least two compression stages, we include investigations on double JPEG compressed images, always reporting higher accuracy than standard approaches.

Forensic analyses of digital images rely heavily on the traces of in-camera and out-camera processes left on the acquired images. Such traces represent a sort of camera fingerprint. If one is able to recover them, by suppressing the high-level scene content and other disturbances, a number of forensic tasks can be easily accomplished. A notable example is the PRNU pattern, which can be regarded as a device fingerprint, and has received great attention in multimedia forensics. In this paper, we propose a method to extract a camera model fingerprint, called noiseprint, where the scene content is largely suppressed and model-related artifacts are enhanced. This is obtained by means of a Siamese network, which is trained with pairs of image patches coming from the same (label +1) or different (label -1) cameras. Although the noiseprints can be used for a large variety of forensic tasks, in this paper we focus on image forgery localization. Experiments on several datasets widespread in the forensic community show noiseprint-based methods to provide state-of-the-art performance.

A problem deeply investigated by multimedia forensics researchers is that of detecting which device has been used to capture a video. This enables us to trace down the owner of a video sequence, which proves extremely helpful to solve copyright infringement cases as well as to fight distribution of illicit material (e.g., child exploitation clips and terroristic threats). Currently, the most promising methods to tackle this task exploit unique noise traces left by camera sensors on acquired images. However, given the recent advancements in motion stabilization of video content, robustness of sensor pattern noise-based techniques is strongly hindered. Indeed, video stabilization introduces geometric transformations to video frames, thus making camera fingerprint estimation problematic with classical approaches. In this paper, we deal with the challenging problem of attributing stabilized videos to their recording device. Specifically, we propose: 1) a strategy to extract the characteristic fingerprint of a device, starting from either a set of images or stabilized video sequences and 2) a strategy to match a stabilized video sequence with a given fingerprint. The proposed methodology is tested on videos coming from a set of different smartphones, taken from the modern publicly available Vision Dataset. The conducted experiments also provide an interesting insight on the effect of modern smartphones video stabilization algorithms on specific video frames.

The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the benchmark is based on Deep-Fakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size. The benchmark is publicly available and contains a hidden test set as well as a database of over 1.8 million manipulated images. This dataset is over an order of magnitude larger than comparable, publicly available, forgery datasets. Based on this data, we performed a thorough analysis of data-driven forgery detectors. We show that the use of additional domain-specific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.

Nonlocal methods are state-of-the-art in SAR despeckling, thanks to their ability to exploit image self-similarity. Given sufficient training data, however, methods based on deep learning have proven highly competitive. Therefore, to take the best of both approaches, we investigate the use of deep learning to improve nonlocal despeckling. We use plain non-iterative nonlocal means despeckling, with weights provided, for each estimation window, by a suitably trained deep CNN. Experiments on synthetic and real SAR data prove this approach to outperform conventional nonlocal methods.

With the availability of powerful and easy-to-use media editing tools, falsifying images and videos has become widespread in the last few years. Coupled with ubiquitous social networks, this allows for the viral dissemination of fake news. This raises huge concerns on multimedia security. This scenario became even worse with the advent of deep learning. New, sophisticated methods have been proposed to accomplish manipulations that were previously unthinkable (e.g., deepfake). This tutorial will present the most reliable methods for detection of manipulated images and for source identification. These are important tools nowadays to carry out fact checking and authorship verification. Hence, this is a timely and relevant research topic in the multimedia security research community.

We propose a new algorithm for the reliable detection and localization of video copy–move forgeries. Discovering well-crafted video copy–moves may be very difficult, especially when some uniform background is copied to occlude foreground objects. To reliably detect both additive and occlusive copy–moves, we use a dense-field approach, with invariant features that guarantee robustness to several postprocessing operations. To limit complexity, a suitable video-oriented version of PatchMatch is used, with a multiresolution search strategy, and a focus on volumes of interest. Performance assessment relies on a new dataset, designedad hoc, with realistic copy–moves and a wide variety of challenging situations. Experimental results show the proposed method to detect and localize video copy–moves with good accuracy even in adverse conditions.

We propose a new method for synthetic aperture radar (SAR) image despeckling, which leverages information drawn from coregistered optical imagery. Filtering is performed by patchwise nonlocal means, working exclusively on SAR data. However, the filtering weights are computed by taking into account also the optical guide, which is much cleaner than the SAR image, and hence more discriminative. To avoid injecting optical-domain information into the filtered image, an SAR-domain statistical test is preliminarily performed to reject right away any risky predictor. Experiments on two SAR-optical data sets prove the proposed method to suppress very effectively the speckle, preserving structural details, and without introducing significant filtering artifacts. Overall, the proposed method compares favorably with all the state-of-the-art despeckling filters, and also with our own previous optical-guided filter.

In this paper, we propose a sparsity-based despeckling approach. The first main contribution of this work is the elaboration of a sparse-coding algorithm adapted to the statistics of SAR images. In fact, most sparse-coding algorithms for SAR data apply a logarithmic transform to data, so as to convert the noise from multiplicative to additive. Then, a Gaussian prior is adopted. However, using a more suitable prior for SAR data avoids introducing artifacts. The second main contribution proposed is to predict the optimal sparsity degree for each patch based on local image features. Experiments show that this strategy improves upon traditional sparse coding with a low-error-rate stopping criterion.

With the widespread diffusion of powerful media editing tools, falsifying images and videos has become easier and easier in the last few years. Fake multimedia, often used to support fake news, represents a growing menace in many fields of life, notably in politics, journalism, and the judiciary. In response to this threat, the signal processing community has produced a major research effort. A large number of methods have been proposed for source identification, forgery detection and localization, relying on the typical signal processing tools. The advent of deep learning, however, is changing the rules of the game. On one hand, new sophisticated methods based on deep learning have been proposed to accomplish manipulations that were previously unthinkable. On the other hand, deep learning provides also the analyst with new powerful forensic tools. Given a suitably large training set, deep learning architectures ensure usually a significant performance gain with respect to conventional methods, and a much higher robustness to post-processing and evasions. In this talk after reviewing the main approaches proposed in the literature to ensure media authenticity, the most promising solutions relying on Convolutional Neural Networks will be explored with special attention to realistic scenarios, such as when manipulated images and videos are spread out over social networks. In addition, an analysis of the efficacy of adversarial attacks on such methods will be presented.

The block-matching 3-D (BM3D) algorithm, based on the nonlocal approach, is one of the most effective methods to date for additive white Gaussian noise image denoising. Likewise, its extension to synthetic aperture radar (SAR) amplitude images, SAR-BM3D, is a state-of-the-art SAR despeckling algorithm. In this paper, we further extend BM3D to address the restoration of SAR interferometric phase images. While keeping the general structure of BM3D, its processing steps are modified to take into account the peculiarities of the SAR interferometry signal. Experiments on simulated and real-world Tandem-X SAR interferometric pairs prove the effectiveness of the proposed method.

Presentation attacks such as using a contact lens with a printed pattern or printouts of an iris can be utilized to bypass a biometric security system. The first international iris liveness competition was launched in 2013 in order to assess the performance of presentation attack detection (PAD) algorithms, with a second competition in 2015. This paper presents results of the third competition, LivDet-Iris 2017. Three software-based approaches to Presentation Attack Detection were submitted. Four datasets of live and spoof images were tested with an additional cross-sensor test. New datasets and novel situations of data have resulted in this competition being of a higher difficulty than previous competitions. Anonymous received the best results with a rate of rejected live samples of 3.36% and rate of accepted spoof samples of 14.71%. The results show that even with advances, printed iris attacks as well as patterned contacts lenses are still difficult for software-based systems to detect. Printed iris images were easier to be differentiated from live images in comparison to patterned contact lenses as was also seen in previous competitions.

Video content can be acquired with off-the-shelf hardware, and is thus increasingly used to record events. With the growing role of video data for communicating to a large audience, we need tools to ensure the authenticity of video content. However, until now, only few methods exist to forensically analyze videos. In this work, we propose a method for statistically comparing two video sequences. Per sequence, intra- and inter-frame residuals are computed. Optical flow is used to compensate for motion artifacts on inter-frame residuals. We use one sequence to build a statistical model, and compare it to the second sequence. From a forensic perspective, the proposed method enables two applications. First, manipulations can be accurately localized if both sequences are subsequences of the same video. Second, source cameras can be distinguished if both sequences stem from different videos. The proposed method is evaluated on collected smartphone data and green-screen splices. Further, it is quantitatively compared to both a recent PRNU-based approach and a technique based on autoencoders.

In this paper we investigate the use of discriminative model learning through Convolutional Neural Networks (CNNs) for SAR image despeckling. The network uses a residual learning strategy, hence it does not recover the filtered image, but the speckle component, which is then subtracted from the noisy one. Training is carried out by considering a large multitemporal SAR image and its multilook version, in order to approximate a clean image. Experimental results, both on synthetic and real SAR data, show the method to achieve better performance with respect to state-of-the-art techniques.

Ship detection is a fundamental task for SAR-based maritime surveillance. Besides providing high reliability, a good detector is required to be computationally light, in order to analyze huge areas in a reasonable time. We propose a fully convolutional neural network for ship detection in SAR images. Thanks to a relatively simple architecture, complexity remains low enough to allow for a single-stage approach, thus avoiding the possible errors of CFAR pre-screening. Experiments on a Sentinel-1 dataset prove the proposed CNN to be much more reliable than CFAR detection.

This work deals with the fusion of SAR and optical data for land cover monitoring. We first propose to use co-registered optical data as a guide for nonlocal SAR image despeckling. Then, we fuse filtered Sentinel-1 SAR data with optical Sentinel-2 data for land-use classification. Experiments show that using optical-driven despeckled SAR data largely improves classification accuracy w.r.t using the original data and even multitemporal filtered data.

Local descriptors based on the image noise residual have proven extremely effective for a number of forensic applications, like forgery detection and localization. Nonetheless, motivated by promising results in computer vision, the focus of the research community is now shifting on deep learning. In this paper we show that a class of residual-based descriptors can be actually regarded as a simple constrained convolutional neural network (CNN). Then, by relaxing the constraints, and fine-tuning the net on a relatively small training set, we obtain a significant performance improvement with respect to the conventional detector.

We devised an algorithm, inspired by contrast-set mining algorithms such as STUCCO, to automatically find statistically significant properties (correlations) in crash groups. Many earlier works focused on improving the clustering of crashes but, to the best of our knowledge, the problem of automatically describing properties of a cluster of crashes is so far unexplored. This means developers currently spend a fair amount of time analyzing the groups themselves, which in turn means that a) they are not spending their time actually developing a fix for the crash; and b) they might miss something in their exploration of the crash data (there is a large number of attributes in crash reports and it is hard and error-prone to manually analyze everything). Our algorithm helps developers and release managers understand crash reports more easily and in an automated way, helping in pinpointing the root cause of the crash. The tool implementing the algorithm has been deployed on Mozilla's crash reporting service.

We propose a despeckling algorithm for multitemporal synthetic aperture radar (SAR) images based on the concepts of block-matching and collaborative filtering. It relies on the nonlocal approach, and it is the extension of SAR-BM3D for dealing with multitemporal data. The technique comprises two passes, each one performing grouping, collaborative filtering, and aggregation. In particular, the first pass performs both the spatial and temporal filtering, while the second pass only the spatial one. To avoid increasing the computational cost of the technique, we resort to lookup tables for the distance computation in the block-matching phases. The experiments show that the proposed algorithm compares favorably with respect to state-of-the-art reference techniques, with better results both on simulated speckled images and on real multitemporal SAR images.

We address the problem of clustering a set of images, according to their source device, in the absence of any prior information. Image similarity is computed based on noise residuals, regarded as single-image estimates of the camera's photo-response non-uniformity (PRNU) pattern. First, residuals are grouped by correlation clustering, and several alternative data partitions are computed as a function of a running decision boundary. Then, these partitions are processed jointly to extract a single, more reliable, consensus clustering and, with it, more reliable PRNU estimates. Finally, both clustering and PRNU estimates are progressively refined by merging pairs of the same-PRNU clusters, selected on the basis of a maximum-likelihood ratio statistic. Extensive experiments prove the proposed method to outperform the current state of the art both on pristine images and compressed images downloaded from social networks. A remarkable feature of the method is that it does not require the user to set any parameter, nor to provide a training set to estimate them. Moreover, through a suitable choice of basic tools, and efficient implementation, complexity remains always quite limited.

We propose a new algorithm for fast approximate nearest neighbor search based on the properties of ordered vectors. Data vectors are classified based on the index and sign of their largest components, thereby partitioning the space in a number of cones centered in the origin. The query is itself classified, and the search starts from the selected cone and proceeds to neighboring ones. Overall, the proposed algorithm corresponds to locality sensitive hashing in the space of directions, with hashing based on the order of components. Thanks to the statistical features emerging through ordering, it deals very well with the challenging case of unstructured data, and is a valuable building block for more complex techniques dealing with structured data. Experiments on both simulated and real-world data prove the proposed algorithm to provide a state-of-the-art performance.

We propose a new approach to SAR despeckling, based on the combination of multiple alternative estimates of the same data. The many despeckling methods proposed in the literature possess different and often complementary strengths and weaknesses. Given a reliable pixel-wise classification of the image, one can take advantage of this diversity by selecting the more appropriate combination of estimators for each image region. We implement a simplified version of this approach, using soft classification and two state-of-the-art despeckling tools, with opposite properties, as basic estimators. Experiments on real-world high-resolution SAR images prove the effectiveness of the proposed technique and confirm the potential of the whole approach.

We propose a new synthetic aperture radar (SAR) despeckling technique based on nonlocal filtering and driven by a coregistered optical image. A preliminary homogeneous versus heterogeneous classification of the image is used to decide where the optical guide can be safely used, thus preventing any distortion of the SAR geometry. Even in regions where the use of optical data is enabled, despeckling is carried out exclusively in the SAR domain, and the optical guide is used only to improve the predictor selection in nonlocal filtering and, hence, in the estimation process. Experiments on real-world imagery confirm the potential of the proposed approach.

A new technique for the segmentation of single- and multiresolution (MR) remote sensing images is proposed. To guarantee the preservation of details at fine scales, edge-based watershed is used, with automatically generated markers that help in limiting oversegmentation. For MR images, the panchromatic and multispectral components are processed independently, extracting both the edge maps and the morphological and spectral markers that are eventually fused at the highest resolution, thus avoiding any information loss induced by pansharpening. Numerical results on object layer extraction and simple classification tasks prove the proposed techniques to provide accurate segmentation maps, which preserve fine details and, contrary to state-of-the-art products, can single out objects equally well at very different scales.

We propose a new algorithm for the accurate detection and localization of copy-move forgeries, based on rotation-invariant features computed densely on the image. Dense-field techniques proposed in the literature guarantee a superior performance with respect to their keypoint-based counterparts, at the price of a much higher processing time, mostly due to the feature matching phase. To overcome this limitation, we resort here to a fast approximate nearest-neighbor search algorithm, PatchMatch, especially suited for the computation of dense fields over images. We adapt the matching algorithm to deal efficiently with invariant features, so as to achieve higher robustness with respect to rotations and scale changes. Moreover, leveraging on the smoothness of the output field, we implement a simplified and reliable postprocessing procedure. The experimental analysis, conducted on databases available online, proves the proposed technique to be at least as accurate, generally more robust, and typically much faster than the state-of-the-art dense-field references.

Biometric authentication systems are quite vulnerable to sophisticated spoofing attacks. To keep a good level of security, reliable spoofing detection tools are necessary, preferably implemented as software modules. The research in this field is very active, with local descriptors, based on the analysis of microtextural features, gaining more and more popularity, because of their excellent performance and flexibility. This paper aims at assessing the potential of these descriptors for the liveness detection task in authentication systems based on various biometric traits: fingerprint, iris, and face. Besides compact descriptors based on the independent quantization of features, already considered for some liveness detection tasks, we will study promising descriptors based on the joint quantization of rich local features. The experimental analysis, conducted on publicly available data sets and in fully reproducible modality, confirms the potential of these tools for biometric applications, and points out possible lines of development toward further improvements.

PRNU-based techniques guarantee a good forgery detection performance irrespective of the specific type of forgery. The presence or absence of the camera PRNU pattern is detected by a correlation test. Given the very low power of the PRNU signal, however, the correlation must be averaged over a pretty large window, reducing the algorithm's ability to reveal small forgeries. To improve resolution, we estimate correlation with a spatially adaptive filtering technique, with weights computed over a suitable pilot image. Implementation efficiency is achieved by resorting to the recently proposed guided filters. Experiments prove that the proposed filtering strategy allows for a much better detection performance in the case of small forgeries.

We propose a new image forgery detection technique which fuses the outputs of two very diverse tools, based on machine learning and block-matching, respectively. The machine-learning tool builds upon some local descriptors recently proposed in the steganalysis field, which are selected and merged based on an ad hoc measure of reliability. The block-matching tool leverages on the patchmatch algorithm for fast search of candidate matchings. Both tools are fine-tuned so as to optimize their fusion which, in turn, exploits the respective strengths and weaknesses of each tool. The proposed technique ranked first in phase 1 of the first Image Forensics Challenge organized in 2013 by the IEEE Signal Processing Society.

We propose an image forgery localization technique which fuses the outputs of three complementary tools, based on sensor noise, machine-learning and block-matching, respectively. To apply the sensor noise tool, a preliminary camera identification phase was required, followed by estimation of the camera fingerprint, and then forgery detection and localization. The machine-learning is based on a suitable local descriptor, while block-matching relies on the PatchMatch algorithm. A decision fusion strategy is then implemented, based on suitable reliability indexes associated with the binary masks. The proposed technique ranked first in phase 2 of the first Image Forensics Challenge organized in 2013 by the IEEE Information Forensics and Security Technical Committee (IFS-TC).

In this work we propose a new algorithm for copy-move forgery detection and localization, based on the fast computation of a dense nearest-neighbor field. To this end, we use PatchMatch, an iterative randomized algorithm for nearest-neighbor search, which exploits the regularity of natural images to converge very rapidly to a near-optimal and smooth field. We modify the basic algorithm to gain robustness against rotations, while keeping the original computational efficiency. Experimental results show the proposed technique to outperform almost uniformly all tested reference techniques in terms of both accuracy and speed.

Digital camera identification is a very active research area, with important applications in the forensics field. Several approaches have been proposed in recent years for this task. One of the most promising is based on the estimation of the sensor noise pattern, used as a sort of camera fingerprint. However, a clever attacker can estimate a camera fingerprint and use it maliciously: this calls for new countermeasures, and so on, in a typical two-party game. In this paper we consider the triangle test, a well-know countermeasure against fake fingerprint attacks, and propose a new algorithm for improving the attacker's success rate. Numerical experiments show that, in typical scenarios, the proposed algorithm improves significantly the attacker performance.

Reliable segmentation of SAR images requires some forms of user supervision: we resort here to the interactive version of the Tree-Structured Markov Random Field (TS-MRF) segmentation suite. The TS-MRF model, and the associated segmentation tool, provide a flexible and spatially adaptive description of the data. In the interactive version, the user can drive the process based on the inspection of the current result, deciding step-by-step which direction to take, and switching from one segmentation modality to another. Experiments with the segmentation and classification of multitemporal SAR images prove the potential of the interactive approach and of the TS-MRF tool.

We address the problem of SAR despeckling by resorting to nonlocal filtering guided by an optical image. In fact, given the increasing availability of remote-sensing optical images, it makes perfect sense trying to use them to improve the performance of despeckling. Our technique exploits the optical image to reliably estimate the statistical similarity among pixels, which is used to evaluate the weights of nonlocal filtering. Optical data are not used to estimate SAR values, but only to guide the overall process. In addition, they are discarded altogether in regions where SAR and optical images present different local geometries, identified by a preliminary classification step, avoiding thus any additional distortion. Experimental results show the proposed approach to provide images of better quality than state-of-the-art conventional filters.

Despeckling techniques based on the nonlocal approach provide an excellent performance, but exhibit also a remarkable complexity, unsuited to time-critical applications. In this letter, we propose a fast nonlocal despeckling filter. Starting from the recent SAR-BM3D algorithm, we propose to use a variable-size search area driven by the activity level of each patch, and a probabilistic early termination approach that exploits speckle statistics in order to speed up block matching. Finally, the use of look-up tables helps in further reducing the processing costs. The technique proposed conjugates excellent performance and low complexity, as demonstrated on both simulated and real-world SAR images and on a dedicated SAR despeckling benchmark.

Objective performance assessment is a key enabling factor for the development of better and better image processing algorithms. In synthetic aperture radar (SAR) despeckling, however, the lack of speckle-free images precludes the use of reliable full-reference measures, leaving the comparison among competing techniques on shaky bases. In this paper, we propose a new framework for the objective (quantitative) assessment of SAR despeckling techniques, based on simulation of SAR images relevant to canonical scenes. Each image is generated using a complete SAR simulator that includes proper physical models for the sensed surface, the scattering, and the radar operational mode. Therefore, in the limits of the simulation models, the employed simulation procedure generates reliable and meaningful SAR images with controllable parameters. Through simulating multiple SAR images as different instances relevant to the same scene we can therefore obtain, a true multilook full-resolution SAR image, with an arbitrary number of looks, thus generating (by definition) the closest object to a clean reference image. Based on this concept, we build a full performance assessment framework by choosing a suitable set of canonical scenes and corresponding objective measures on the SAR images that consider speckle suppression and feature preservation. We test our framework by studying the performance of a representative set of actual despeckling algorithms; we verify that the quantitative indications given by numerical measures are always fully consistent with the rationale specific of each despeckling technique, strongly agrees with qualitative (expert) visual inspections, and provide insight into SAR despeckling approaches.

Graphics editing programs of the last generation provide ever more powerful tools, which allow for the retouching of digital images leaving little or no traces of tampering. The reliable detection of image forgeries requires, therefore, a battery of complementary tools that exploit different image properties. Techniques based on the photo-response non-uniformity (PRNU) noise are among the most valuable such tools, since they do not detect the inserted object but rather the absence of the camera PRNU, a sort of camera fingerprint, dealing successfully with forgeries that elude most other detection strategies. In this paper, we propose a new approach to detect image forgeries using sensor pattern noise. Casting the problem in terms of Bayesian estimation, we use a suitable Markov random field prior to model the strong spatial dependences of the source, and take decisions jointly on the whole image rather than individually for each pixel. Modern convex optimization techniques are then adopted to achieve a globally optimal solution and the PRNU estimation is improved by resorting to nonlocal denoising. Large-scale experiments on simulated and real forgeries show that the proposed technique largely improves upon the current state of the art, and that it can be applied with success to a wide range of practical situations.

The estimation of the fractal dimension of a natural surface using spectral analysis of amplitude SAR data is a powerful tool for geophysical applications. However, the obtained estimates are influenced by the presence of speckle. In this paper we present first results regarding the analysis of the effects on fractal dimension maps due to the application of despeckling techniques on SAR images. The use of simulated data allows to obtain also multilook images, presenting a very high number of looks, which should represent a virtually speckle-free image. In the results section different despeckling techniques are considered, relaying on both classical spatial filtering and non-local means. The presented results allow to draw meaningful conclusions on the effects of speckle and despeckling on spectral estimation and fractal dimension retrieving.

Detection of image forgeries is an important topic for forensics applications. One of the most interesting approaches to forgery detection relies on the photo-response non uniformity noise (PRNU), that can be considered as a sort of camera fingerprint and used as such to accomplish forgery detection. In fact, while genuine parts of an image exhibit the camera PRNU, this is not present in tampered areas. In this work, we present a new method to detect forgeries by using PRNU. In particular, we propose a minimum-risk Bayesian classification, aimed at minimizing the probability of error or, more in general, a weighted average of the two types of errors (false alarm, missing detection) according to their importance for the application. Then, we introduce a regularization term in the decision process to take into account prior information on the classification map. This step weights optimally the observed data and the regularity constraints to minimize the Bayesian risk. Since the regularization term is based on spatial properties of the decision map, classification cannot work on each pixel individually but must be carried out jointly on the whole image. The ensuing problem is NP-hard but we use relaxation and convex optimization techniques, based on proximal methods, to obtain a global optimum solution in limited time. Preliminary experiments with digital forgeries of different sizes and shapes prove that the improved technique provides a significant performance gain w.r.t. the original, at the cost of a limited increase in complexity.

We propose a new framework for the quantitative assessment of SAR despeckling techniques, based on physical-level simulation of SAR images corresponding to canonical scenes. Thanks to the simulator, we can generate multiple SAR images of the same scene which differ only in the speckle content, and, hence, a true multilook SAR image, with an arbitrarily large number of looks, to use as “speckle-free” reference. Based on this concept, we select a small set of canonical scenes and, for each of them, a suitable set of objective measures which account for speckle suppression and image feature preservation. We gain insight into the system reliability by comparing the indications it gives for some sample despeckling filters with those obtained by expert visual inspection of the filtered images.

We propose a novel despeckling algorithm for synthetic aperture radar (SAR) images based on the concepts of nonlocal filtering and wavelet-domain shrinkage. It follows the structure of the block-matching 3-D algorithm, recently proposed for additive white Gaussian noise denoising, but modifies its major processing steps in order to take into account the peculiarities of SAR images. A probabilistic similarity measure is used for the block-matching step, while the wavelet shrinkage is developed using an additive signal-dependent noise model and looking for the optimum local linear minimum-mean-square-error estimator in the wavelet domain. The proposed technique compares favorably w.r.t. several state-of-the-art reference techniques, with better results both in terms of signal-to-noise ratio (on simulated speckled images) and of perceived image quality.

Speckle reduction is a key step in several SAR image processing procedures. In this paper, a new despeckling technique based on the “nonlocal” denoising filter BM3D [1] is presented. The filter has been modified in order to take into account SAR image characteristics. The experimental results, conducted on both synthetic and real SAR images, confirm the potential of the proposed approach.

In this work we propose a modified version of the BM3D algorithm recently introduced by Dabov et al. for the denoising of images corrupted by additive white Gaussian noise. The original technique performs a multipoint filtering, where the nonlocal approach is combined with the wavelet shrinkage of a 3D cube composed by similar patches collected by means of block-matching. Our improvement concerns the thresholding of wavelet coefficients, which are subject to a different shrinkage depending on their level of sparsity. The modified algorithm is more robust with respect to block matching errors, especially when noise is high, as proved by experimental results on a large set of natural images.

The contourlet transform was recently proposed to overcome the limited ability of wavelet to represent image edges and discontinuities. Besides retaining the desirable characteristics of wavelet transform, such as multiresolution and localization, it has two additional important features: directionality and anisotropy. In this work we propose a new image coding technique based on an hybrid contourlet-wavelet decomposition. The encoder builds upon the well-known SPIHT algorithm, which is suitably modified to take into account the new hierarchical structure of the transform coefficients and the correlation between them. Both numerical results and visual quality confirm the potential of this approach, especially for images with high texture content.

In this letter, we improve the class-based transform-coding scheme proposed by Gelli and Poggi for the compression of multispectral images. The original spatial-coding tools, 1-D discrete cosine transform and scalar quantization, are replaced by shape-adaptive wavelet transform and set partitioning in hierarchical trees. Numerical experiments show that the improved technique outperforms the original one for medium- to high-quality compression and is consistently superior to all reference techniques.

We propose a new efficient region-based scheme for the compression of multispectral remote-sensing images. The region-based description of an image comprises a segmentation map, which singles out the relevant regions and provides their main features, followed by the detailed (possibly lossless) description of each region. The map conveys information on the image structure and could even be the only item of interest for the user; moreover, it enables the user to perform a selective download of the regions of interest, or can be used for high-level data mining and retrieval applications. This approach, with the multiple pieces of information required, may seem inherently inefficient. The goal of this research is to show that, by carefully selecting the appropriate segmentation and coding tools, region-based compression of multispectral images can be also effective in a rate-distortion sense, thus providing an image description that is both insightful and efficient. To this end, we define a generic coding scheme, based on Bayesian image segmentation and on transform coding, where several key design choices, however, are left open for optimization, from the type of transform, to the rate allocation procedure, and so on. Then, through an extensive experimental phase on real-world multispectral images, we gain insight on such key choices, and finally single out an efficient and robust coding scheme, with Bayesian segmentation, class-adaptive Karhunen-Loève spectral transform, and shape-adaptive wavelet spatial transform, which outperforms state-of-the-art and carefully tuned conventional techniques, such as JPEG-2000 multicomponent or SPIHT-based coders.

The region-based description of multispectral images enables important high-level tasks such as data mining and retrieval, and region-of-interest selection. In order to obtain an efficient representation of such images we resort to adaptive transform coding techniques. Such techniques, however, require a considerable information overhead, which must be carefully managed to obtain a satisfactory rate-distortion performance. In this work we develop several region-based coding schemes and compare them with conventional (non-adaptive) and class-based schemes, so as to single out the rate-distortion gains/losses of this approach.

In this work we implement and compare several state-of-the-art transform coding schemes for the compression of multispectral images, in order to better understand which elements have a deeper impact on the overall performance, and which tools guarantee the best results. All schemes are based on Karhunen-Loeve transform and/or wavelet transform, in various combinations, and use SPIHT as the coding engine. Moreover, besides the ordinary techniques, their object-based counterparts are also examined, so as to study the viability of such approach [M. Cagnazzo et al., Oct 2004] for these images. Whenever possible, an optimal rate allocation strategy is applied. The experiments, performed on images acquired by two different sensors, highlight the superiority of KLT as spectral transform; the rough equivalence between object-based and ordinary techniques in terms of rate-distortion performance; and the importance of the optimal allocation.

Region-based encoding techniques have been long investigated for the compression of still images and video sequences and have recently gained much popularity, as testified by the object-based nature of the MPEG-4 video coding standard. This work aims at analyzing costs and advantages of implementing such an approach by shape-adaptive wavelet transform and shape-adaptive SPIHT. The analysis of several performance measures in a number of experiments confirm the potential of wavelet-based region-based approach, and provide insight about what performance gains and losses can be expected in various operative conditions.

We present a new technique for the compression of remote-sensing hyperspectral images based on wavelet transform and zerotree coding of coefficients. In order to improve encoding efficiency, the image is first segmented in a small number of regions with homogeneous texture. Then, a shape-adaptive wavelet transform is carried out on each region and the resulting coefficients are finally encoded by a shape-adaptive version of SPIHT. Thanks to the segmentation map (sent as a side information) region boundaries are faithfully preserved and selective encoding strategies can be easily implemented. In addition, by-now homogeneous region textures can be more efficiently encoded.

Multitemporal remote sensing images are useful tools for many applications in natural resource management. Compression of this kind of data is an issue of interest, yet, only a few paper address it specifically, while general-purpose compression algorithms are not well suited to the problem, as they do not exploit the strong correlation among images of a multitemporal set of data. Here we propose a coding architecture for multitemporal images, which takes advantage of segmentation in order to compress data. Segmentation subdivides images into homogeneous regions, which can be efficiently and independently encoded. Moreover this architecture provides the user with a great flexibility in transmitting and retrieving only data of interest

SAR images are severely degraded by speckle, and filtering is therefore a common practice. Filtering is especially useful before compression, to avoid spending valuable resources to represent noise; unfortunately, it also degrades important image features, like region boundaries. To overcome this problem, one can resort to a segmentation-based compression scheme, which allows one to preserve region boundaries, carry out intense denoising, and improve overall performance. In this work we assess the potential of segmentation-based compression through controlled experiments on synthetic SAR images. Numerical results seem to confirm the validity of this approach.

There is an ever-growing interest in the compression of SAR data because of the huge resources required for storage and transmission. This is especially true for spaceborne sensors, given the limited capacity of the downlink channel. Unfortunately, SAR data lack the useful properties on which compression algorithms rely; indeed, these are present in the focused images, but focusing is too complex for on-board implementation at this time. Poggi et al. (2000) proposed to perform on the satellite only the low-complexity range focusing, which increases the data correlation and better concentrates their energy. These properties were then exploited by adopting a variable-rate vector quantizer, with a clear performance improvement with respect to reference techniques. However, vector quantization (VQ) is too complex for actual on-board implementation, and therefore, here we replace VQ with trellis-coded VQ. To limit complexity, only small vectors are used, which reduces VQ's ability to exploit data dependencies; on the other hand, trellis coding allows one to encode large blocks of data at once, and to obtain a better partition of the input space. Experiments on real SAR data show that the overall performance is comparable to that of Poggi et al., but the complexity is much lower, making on-board implementation possible.

In spaceborne SAR systems, some form of data compression is required to reduce the bandwidth of the downlink channel. Compressing the raw data is very inefficient because such data exhibit little or no correlation. Data focusing would restore the original dependencies of the image, but it is too complex to be implemented onboard. In this paper we propose to carry out a partial, low-complexity focusing onboard, and then compress the resulting data by means of a suitable algorithm based on variable-rate adaptive vector quantization. Experiments show a performance improvement of 0.5-0.7 dB over compression techniques proposed in the literature.