We propose a model to represent quasi-flat objects, such as ancient coins. These objects are flat surfaces, meaning their length and their width largely exceed their height, and feature a distinctive relief. This relief characterizes the object and its perception is directly influenced by the position of the object, the light direction and the viewer’s direction. Our model is a single (non classic) image representation containing the underlying structural variations of the object. This model, that we call “Multi-Light Energy Map”, is constructed out of several classic images taken with several illumination directions without computing the object’s surface normals. Together with this information about the magnitude of the object variations, it is possible to compute the information about the angular part, that we call “Multi-Light Orientation Map”. Using these two maps, it is possible to render an image of the object at any light azimuth, enabling the user to move the light sources during the visualization of the object. Moreover, these maps can be useful for the retrieval of ancient coins, either by robust image registration, or by object recognition using either contours or textures.

In cases of digital enrolment via mobile and online services, identity documents (IDs) verification is critical to efficiently detect forgery and therefore build user trust in the digital world. In this paper, we propose a copy-move public dataset, called FMIDV (forged mobile ID video dataset) containing forged IDs with respect to guilloche patterns. Also, we propose two fraud detection models on guilloche patterns of IDs, which are based on contrastive and adversarial learning. In the sequel, each proposed model manages to read the entire ID and to recognize the guilloche pattern to check its similarity to the pattern of an authentic ID. The objective of the similarity check is to validate its authenticity or its rejection. Experiments are conducted on MIDV and FMIDV datasets to analyze and identify the most proper parameters to achieve higher authentication performance. The code and the dataset are available at https://github.com/malghadi/CheckID.

One of important challenges in the document liveness detection process for identity document verification is quality verification. To tackle this challenge, this paper proposes a reference hashing approach to discriminate between the original template of the identity document image and the scan one, which is called CheckScan. Actually, the discrimination process takes place between two aligned identity document images. The proposed approach is made up of two steps: feature extraction based on Fast Fourier Transform (FFT) and hash construction. Feature extraction step involves partitioning the identity document image into set of non-overlapping blocks, and for each block the FFT magnitude spectrum is calculated. Hence, a specific number from the FFT magnitude peaks is selected as discriminative features. The hash construction step quantizes the selected peaks into binary codes by applying a new quantization approach that is based on the coordinates of the selected peaks. These two steps are combined together in this work to achieve good discriminate (well anti-collision) capability for distinct identity document images. Experiments were conducted in order to analyze and identify the most proper parameters to achieve higher discrimination performance. The experimental results were performed on the Mobile Identity Document Video dataset (MIDV-2020), and the results show that the proposed approach builds binary codes quite discriminative for distinct identity document images.

In this paper, we propose a compliant scheme of image hashing that is based on gradient measurement. The proposed approach relies on a local geometry variation analysis in multi-channel images. Derivative filters are used to estimate the image gradient and a suitable representation of geometric color features is introduced. A gradient norm is provided as an intermediate hash, so we compute subsequently the magnitude of the Fourier spectrum in order to provide a final hash. The aim is to preserve accurately the structural information along with smoothing operations. We conduct experiments on the CASIA-V2 database for the tampering detection task, and the UCID database to demonstrate the hash robustness. We use the metrics of true positive rate (TPR) and false positive rate (FPR) to investigate the performance of the proposed method. A comparison is carried out using two corresponding schemes; the first operates on the quaternion discrete Fourier transform (QDFT) to take into account the image color planes, and the second exploits this transform into the log-polar domain. According to the TPR results, our method is quite robust against different content-preserving operations applied on the UCID database with regard to a predefined threshold. The FPR results over CASIA-V2 further demonstrate a superior capability of the proposed approach in detecting image forgeries.

In this paper, a novel local threshold binarization method using fast Fuzzy C-Means clustering is proposed. Historical document images with non-uniform background, stains, faded ink are first processed by removing the background using inpainting based method. Then using Fuzzy C-Means clustering is used to cluster out the pixels into three main clusters : sure text pixels, sure background pixels and confused pixels which may or may not be labeled as text. Based on the structural symmetry of pixels (SSP), these confused pixels are then classified into text or background pixels. The SSP is defined as those pixels around strokes whose gradient magnitudes are big enough and whose directions are opposite. As the gradient map is our basis for computing the SSP, we further propose to estimate the background surface first and to extract potential SSP in the compensated image so as to deal with degradations of document images such as uneven illumination, low contrast and stain. To prove the effectiveness of our method, tests on eight public document image datasets are preformed and the experimental results show that our method outperforms other local threshold binarization approaches on both F-measure and PSNR.

As the adoption of digital technologies expands, it becomes vital to build trust and confidence in the integrity of such technology. The SPIRIT project investigates the proof of concept of employing novel secure and privacy-ensuring techniques in services set-up in the Internet of Things (IoT) environment, aiming to increase the trust of users in IoTbased systems. The proposed system integrates three highly novel technology concepts developed by the consortium partners. Specifically, a technology, ermed ICMetrics, for deriving encryption keys directly from the operating characteristics of digital devices; secondly, a technology based on a contentbased signature of user data in order to ensure the integrity of sentdata upon arrival; a third technology, termed semantic firewall, which is able to allow or deny the transmission of data derived from an IoT device according to the information contained within the data and the information gathered about the requester.

In the context of document security systems, there is a growing need for a stable segmentation method. State-of-the-art document image segmentation methods are not stable as they use several parameters and thresholds such as binarization. Hence, this paper presents a new method for segmentation based on a new definition of connected color components and a new model of human vision. Our algorithm produces results that are three to four times more stable than state-of-the-art superpixel segmentation methods while maintaining a similar segmentation accuracy.

In order to create an automatic document security system one needs to secure the textual content but also the graphical content of the document. This paper proposes a hashing algorithm capable of securing the graphical parts of paper and digital documents with unprecedented performance and a very small digest. The main challenge for such an algorithm is that of stability, in particular with respect to print and scan noise. We define the generic notion of stability and how to evaluate it. To achieve such performance we use both dense local information and global descriptors. We have tested our method on two datasets totaling nearly 45000 images.

The importance of having stable information extraction algorithms for security related applications and more generally for industrial use cases has been recently highlighted. Stability is what makes an algorithm reliable as it gives a guarantee that the results will be reproducible on similar data. Without it, security criteria such as the probability of false positives cannot be quantified. As a consequence, no security application can be built from an unstable algorithm. In a document verification framework, the probability of false positives indicates the probability that two different results are given for two copies of the same document. This paper builds on our previous work about a stable layout descriptor to study the stability of four segmentation algorithms. We consider that a segmentation algorithm is stable if it produces the same layout for all copies of the same document. The algorithms studied are two versions of PAL, Voronoi, and JSEG. We compare the stability of the different algorithms and study the factors influencing their stability.

Security applications related to document authentication require an exact match between an authentic copy and the original of a document. This implies that the documents analysis algorithms that are used to compare two documents (original and copy) should provide the same output. This kind of algorithm includes the computation of layout descriptors from the segmentation result, as the layout of a document is a part of its semantic content. To this end, this paper presents a new layout descriptor that significantly improves the state of the art. The basic of this descriptor is the use of a Delaunay triangulation of the centroids of the document regions. This triangulation is seen as a graph and the adjacency matrix of the graph forms the descriptor. While most layout descriptors have a stability of 0% with regard to an exact match, our descriptor has a stability of 74% which can be brought up to 100% with the use of an appropriate matching algorithm. It also achieves 100% accuracy and retrieval in a document retrieval scheme on a database of 960 document images. Furthermore, this descriptor is extremely efficient as it performs a search in constant time with respect to the size of the document database and it reduces the size of the index of the database by a factor 400.

Nowadays, document forgery is becoming a real issue. A large amount of documents that contain critical information as payment slips, invoices or contracts, are constantly subject to fraudster manipulation because of the lack of security regarding this kind of document. Previously, a system to detect fraudulent documents based on its intrinsic features has been presented. It was especially designed to retrieve copy-move forgery and imperfection due to fraudster manipulation. However, when a set of characters is not present in the original document, copy-move forgery is not feasible. Hence, the fraudster will use a text toolbox to add or modify information in the document by imitating the font or he will cut and paste characters from another document where the font properties are similar. This often results in font type errors. Thus, a clue to detect document forgery consists of finding characters, words or sentences in a document with font properties different from their surroundings. To this end, we present in this paper an automatic forgery detection method based on document font features. Using the Conditional Random Field a measurement of probability that a character belongs to a specific font is made by comparing the character font features to a knowledge database. Then, the character is classified as a genuine or a fake one by comparing its probability to belong to a certain font type with those of the neighboring characters.

Current security applications rely on the performances of the algorithms that they use. For document authentication, document analysis algorithms should be precise enough to detect any modification. They should also be stable enough so that a document and its photocopy yield the same result. This requirement is an absolute stability. Having close values is not enough. They need to be exactly the same. This paper presents our preliminary work on the case of a stable layout descriptor. While everyone knows that thresholds are a source of instability, they are still common practice. We describe a promising layout descriptor which drastically reduces the number of thresholds compared to the state of the art. Unfortunately, it is not stable enough when tested on real data. There are still too many thresholds. This paper opens and justifies the path towards algorithms without any threshold.

This article tackles some important issues relating to the analysis of a particular case of complex ancient graphic images, called “lettrines”, “drop caps”, or “ornamental letters”. Our contribution focuses on proposing generic solutions for lettrine recognition and classification. Firstly, we propose a bottom-up segmentation method, based on texture, ensuring the separation of the letter from the elements of the background in an ornamental letter. Secondly, a structural representation is proposed for characterizing a lettrine. This structural representation is based on filtering automatically relevant information by extracting representative homogeneous regions from a lettrine to generate a graph-based signature. The proposed signature provides a rich and holistic description of the lettrine style by integrating varying low-level features (e.g. texture). Then, to categorize and classify lettrines with similar style, structure (i.e. ornamental background) and content (i.e. letter), a graph-matching paradigm has been carried out to compare and classify the resulting graph-based signatures. Finally, to demonstrate the robustness of the proposed solutions and provide additional insights into their accuracies, an experimental evaluation has been conducted using a relevant set of lettrine images. In addition, we compare the results achieved with those obtained using the state-of-the-art methods to illustrate the effectiveness of the proposed solutions.

The work conducted in this article presents a structural signature based on texture for the characterization and categorization of digitized historical book pages. The proposed signature does not assume a priori knowledge regarding page layout and content, and hence, it is applicable to a large variety of ancient books. By integrating varying low-level features (e.g. texture) characterizing the different page components (i.e. different text fonts, or graphic regions) on the one hand, and structural information describing the page layout on the other hand, the proposed signature provides a rich and holistic description of the layout and content of the analyzed book pages. More precisely, the signature-based characterization approach consists of two stages. The first stage is extracting automatically homogeneous regions. Then, the second one is proposing a graph-based page signature, which is based on the extracted homogeneous regions, reflecting its layout and content. This signature ensures the implementation of numerous applications for managing effectively a corpus or collections of books (e.g. information retrieval in digital libraries according to several criteria, or page categorization). To illustrate the effectiveness of the proposed page signature, a detailed experimental evaluation has been conducted in this article for assessing two possible categorization applications, unsupervised page classification and page stream segmentation.

For the segmentation of ancient digitized document images, it has been shown that texture feature analysis is a consistent choice for meeting the need to segment a page layout under significant and various degradations. In addition, it has been proven that the texture-based approaches work effectively without hypothesis on the document structure, neither on the document model nor the typographical parameters. Thus, by investigating the use of texture as a tool for automatically segmenting images, we propose to search homogeneous and similar content regions by analyzing texture features based on a multiresolution analysis. The preliminary results show the effectiveness of the texture features extracted from the autocorrelation function, the Grey Level Co-occurrence Matrix (GLCM), and the Gabor filters. In order to assess the robustness of the proposed texture-based approaches, images under numerous degradation models are generated and two image enhancement algorithms (non-local means filtering and superpixel techniques) are evaluated by several accuracy metrics. This study shows the robustness of texture feature extraction for segmentation in the case of noise and the uselessness of a demising step.

Example-based super-resolution (SR) methods learn the correspondences between low resolution (LR) and high-resolution (HR) image patches, where the patches are extracted from a training database. To reconstruct a single LR image into a HR one, each LR image patch is processed by the previously trained model to recover its corresponding HR patch. For this reason, they are computationally inefficient. We propose the use of a selective patch processing technique to carry out the super-resolution step more efficiently, while maintaining the output quality. In this technique, only patches of high variance are processed by the costly reconstruction steps, while the rest of the patches are processed by fast bicubic interpolation. We have applied the proposed improvement on representative example-based SR methods to super-resolve text images. The results show a significant speed up for text SR without a drop in theocrat accuracy. In order to carry out an extensive and solid performance evaluation, we also present a public database of text images for training and testing example-based SR methods.

In video-surveillance, non-parametric Bayesian approaches based on a Hierarchical Dirichlet Process (HDP) have recently shown their efficiency for modeling crowed scene activities. This paper follows this track by proposing a method for detecting and clustering emergent behaviors across different captures made of numerous unconstrained trajectories. Most HDP applications for crowed scenes (e.g. traffic, pedestrians) are based on flow motion features. In contrast, we propose to tackle the problem by using full individual trajectories. Furthermore, our proposed approach relies on a three-level clustering hierarchical Dirichlet process able with a minimum a priori to hierarchically retrieve behaviors at increasing semantical levels: activity atoms, activities and behaviors. We chose to validate our approach on ant trajectories simulated by a Multi-Agent System (MAS) using an ant colony foraging model. The experimentation results have shown the ability of our approach to discover different emergent behaviors at different scales, which could be associated to observable events such as "forging" or "deploying" for instance.

Recently, texture-based features have been used for digitized historical document image segmentation. It has been proven that these methods work effectively with no a priori knowledge. Moreover, it has been shown that they are robust when they are applied on degraded documents under different noise levels and types. In this paper an approach of evaluating texture-based feature sets for segmenting historical documents is presented in order to compare them. We aim at determining which texture features could be more adequate for segmenting graphical regions from textual ones on the one hand and for discriminating text in a variety of situations of different fonts and scales on the other hand. For this purpose, six well-known and widely used texture-based feature sets (autocorrelation function, Grey Level Co occurrence Matrix, Gabor filters, 3-level Haar wavelet transform, 3-level wavelet transform using 3-tap Daubechies filter and 3-level wavelet transform using 4-tap Daubechies filter) are evaluated and compared on a large corpus of historical documents. An additional insight into the computation time and complexity of each texture-based feature set is given. Qualitative and numerical experiments are also given to demonstrate each texture-based feature set performance.

Blur is one of the most difficult distortions in camera captured documents. It degrades the visual quality of an image, and makes it difficult to read whether by a human or OCR systems. This paper presents a novel non-blind deblurring method that combines the well known effective techniques of sparse representations and non-local image similarity. The presented problem formulation enables the use of standard sparse coding methods for solving sparse coding-based deblurring when enhanced by a non-local means prior. The method has been tested on both synthetic and real document images degraded with a variety of blur kernels. The resulting deblurred images have high quality in terms of both signal-to-noise ratio and OCR accuracy.

International audience

Paper documents still represent a large amount of information supports used nowadays and may contain critical data. Even though official documents are secured with techniques such as printed patterns or artwork, paper documents suffer from a lack of security. However, the high availability of cheap scanning and printing hardware allows non-experts to easily create fake documents. As the use of a watermarking system added during the document production step is hardly possible, solutions have to be proposed to distinguish a genuine document from a forged one. In this paper, we present an automatic forgery detection method based on document's intrinsic features at character level. This method is based on the one hand on outlier character detection in a discriminant feature space and on the other hand on the detection of strictly similar characters. Therefore, a feature set is computed for all characters. Then, based on a distance between characters of the same class, the character is classified as a genuine one or a fake one.

In the context of historical collection conservation and worldwide diffusion, this paper presents an automatic approach of historical book page layout segmentation. In this article, we propose to search the homogeneous regions from the content of historical digitized books with little a priori knowledge by extracting and analyzing texture features. The novelty of this work lies in the unsupervised clustering of the extracted texture descriptors to find homogeneous regions, i.e. graphic and textual regions, by performing the clustering approach on an entire book instead of processing each page individually. We propose firstly to characterize the content of an entire book by extracting the texture information of each page, as our goal is to compare and index the content of digitized books. The extraction of texture features, computed without any hypothesis on the document structure, is based on two non-parametric tools: the autocorrelation function and multiresolution analysis. Secondly, we perform an unsupervised clustering approach on the extracted features in order to classify automatically the homogeneous regions of book pages. The clustering results are assessed by internal and external accuracy measures. The overall results are quite satisfying. Such analysis would help to construct a computer-aided categorization tool of pages.

In this paper, we present a super-resolution (SR) method based on iterative backprojections. Motion information is used for the synthesis of the restoration filter in the SR method. Both, the blur estimation and the choice of the degradation model, are based on estimated global motion. The method is applied to highly under-sampled images such as DC images of MPEG compressed video and medical magnetic resonance (MR) images. Results of the comparison of our blur model with some common blur models are encouraging.

This article presents a scene similarity measure for video content segmentation. In the context of the rough indexing paradigm, we extract only partial information from MPEG compressed streams to measure the similarity of video frames through time. The similarity measure of I-Frames is defined based on motion compensation of DC images and local contrast computation. The method allows a real-time segmentation of the video content. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 765–783, 2006.

In this paper, we describe a method for the construction of super-resolution (SR) mosaics. The low-resolution (LR) input sequence of the SR algorithm consists of DC images of I-frames (DCI-frames) extracted from MPEG compressed streams. For the registration of the LR sequence, the motion information from P-frames is used. The novelty of this approach is the determination of the optical transfer function (OTF) of the blur for the image restoration in the SR algorithm. First results are promising.