In this paper, we present a new approach to model a printing-imaging channel using a machine learning-based “digital twin” for copy detection patterns (CDP). The CDP are considered as modern anti-counterfeiting features in multiple applications. Our digital twin is formulated within the information-theoretic framework of TURBO initially developed for high energy physics simulations, using variational approximations of mutual information for both encoder and decoder in the bidirectional exchange of information. This model extends various architectural designs, including paired pix2pix and unpaired CycleGAN, for image-to-image translation. Applicable to any type of printing and imaging devices, the model needs only training data comprising digital templates sent to a printing device and data acquired by an imaging device. The data can be paired, unpaired, or hybrid, ensuring architectural flexibility and scalability for multiple practical setups. We explore the influence of various architectural factors, metrics, and discriminators on the overall system’s performance in generating and predicting printed CDP from their digital versions and vice versa. We also performed a comparison with several state-of-the-art methods for image-to-image translation applications. The simulation code and extended results are publicly available at https://gitlab.unige.ch/sip-group/digital-twin.

Copy Detection Pattern (CDP) technology is a promising anti-counterfeiting solution for the protection of physical goods. In recent years, it has been shown that this technology is threatened by powerful deep learning attacks that are able to bypass original authentication schemes. In this paper, we tackle this problem by proposing a new CDP authentication scheme based on statistical knowledge discovered about the printing and imaging process. The novelty of our approach lies in providing means to measure the reliability of each local pattern appearing in the CDP. This allows to define new authentication measures to better differentiate original CDP from fakes. Our results show that this new system is capable of performing reliable CDP authentication with smartphones without the need for heavyweight machine learning tools requiring massive data entries.

In the recent years, the copy detection patterns (CDP) attracted a lot of attention as a link between the physical and digital worlds, which is of great interest for the internet of things and brand protection applications. However, the security of CDP in terms of their reproducibility by unauthorized parties or clonability remains largely unexplored. In this respect, this paper addresses a problem of anti-counterfeiting of physical objects and aims at investigating the authentication aspects and the resistances to illegal copying of the modern CDP from machine learning perspectives. A special attention is paid to a reliable authentication under the real-life verification conditions when the codes are printed on an industrial printer and enrolled via modern mobile phones under regular light conditions. The theoretical and empirical investigation of authentication aspects of CDP is performed with respect to four types of copy fakes from the point of view of (i) multi-class supervised classification as a baseline approach and (ii) one-class classification as a real-life application case. The obtained results show that the modern machine-learning approaches and the technical capacities of modern mobile phones allow to reliably authenticate CDP on end-user mobile phones under the considered classes of fakes.

In this paper, we consider privacy-preserving compressed image sharing, where the goal is to release compressed data whilst satisfying some privacy/secrecy constraints yet ensuring image reconstruction with a defined fidelity. The privacy-preserving compressed image sharing is addressed using a machine learning framework based on an information bottleneck with a shared secret key for authorized users. In contrast, an adversary observing the protected compressed representation tries to either reconstruct the data or deduce some privacy-sensitive attributes such as gender, age, etc. The inference task on the adversary’s side is performed without the knowledge of the shared secret key and is based on an adversarial mutual information maximization between the privacy-protected compressed representation and targeted attributes. The proposed framework is experimentally validated on the CelebA dataset.

Copy detection patterns (CDP) are an attractive technology that allows manufacturers to defend their products against counterfeiting. The main assumption behind the protection mechanism of CDP is that these codes printed with the smallest symbol size (1x1) on an industrial printer cannot be copied or cloned with sufficient accuracy due to data processing inequality. However, previous works have shown that Machine Learning (ML) based attacks can produce high-quality fakes, resulting in decreased accuracy of authentication based on traditional feature-based authentication systems. While Deep Learning (DL) can be used as a part of the authentication system, to the best of our knowledge, none of the previous works has studied the performance of a DL-based authentication system against ML-based attacks on CDP with 1x1 symbol size. In this work, we study such a performance assuming a supervised learning (SL) setting.

We propose a practical framework to address the problem of privacy-aware image sharing in large-scale setups. We argue that, while compactness is always desired at scale, this need is more severe when trying to furthermore protect the privacy-sensitive content. We therefore encode images, such that, from one hand, representations are stored in the public domain without paying the huge cost of privacy protection, but ambiguated and hence leaking no discernible content from the images, unless a combinatorially-expensive guessing mechanism is available for the attacker. From the other hand, authorized users are provided with very compact keys that can easily be kept secure. This can be used to disambiguate and reconstruct faithfully the corresponding access-granted images. We achieve this with a convolutional autoencoder of our design, where feature maps are passed independently through sparsifying transformations, providing multiple compact codes, each responsible for reconstructing different attributes of the image. The framework is tested on a large-scale database of images with public implementation available.

This paper addresses a problem of anti-counterfeiting of physical objects and aims at investigating a possibility of counterfeited printable graphical code detection from a machine learning perspectives. We investigate a fake generation via two different deep regeneration models and study the authentication capacity of several discriminators on the data set of real printed graphical codes where different printing and scanning qualities are taken into account. The obtained experimental results provide a new insight on scenarios, where the printable graphical codes can be accurately cloned and could not be distinguished.

In recent years, classification techniques based on deep neural networks (DNN) were widely used in many fields such as computer vision, natural language processing, and self-driving cars. However, the vulnerability of the DNN-based classification systems to adversarial attacks questions their usage in many critical applications. Therefore, the development of robust DNN-based classifiers is a critical point for the future deployment of these methods. Not less important issue is understanding of the mechanisms behind this vulnerability. Additionally, it is not completely clear how to link machine learning with cryptography to create an information advantage of the defender over the attacker. In this paper, we propose a key-based diversified aggregation (KDA) mechanism as a defense strategy in a gray- and black-box scenario. KDA assumes that the attacker (i) knows the architecture of classifier and the used defense strategy, (ii) has an access to the training data set, but (iii) does not know a secret key and does not have access to the internal states of the system. The robustness of the system is achieved by a specially designed key-based randomization. The proposed randomization prevents the gradients' back propagation and restricts the attacker to create a "bypass" system. The randomization is performed simultaneously in several channels. Each channel introduces its own randomization in a special transform domain. The sharing of a secret key between the training and test stages creates an information advantage to the defender. Finally, the aggregation of soft outputs from each channel stabilizes the results and increases the reliability of the final score. The performed experimental evaluation demonstrates a high robustness and universality of the KDA against state-of-the-art gradient-based gray-box transferability attacks and the non-gradient-based black-box attacks (The results reported in this paper have been partially presented in CVPR 2019 (Taran et al., Defending against adversarial attacks by randomized diversification, 2019) & ICIP 2019 (Taran et al., Robustification of deep net classifiers by key-based diversified aggregation with pre-filtering, 2019)).

The vulnerability of machine learning systems to adversarial attacks questions their usage in many applications. In this paper, we propose a randomized diversification as a defense strategy. We introduce a multi-channel architecture in a gray-box scenario, which assumes that the architecture of the classifier and the training data set are known to the attacker. The attacker does not only have access to a secret key and to the internal states of the system at the test time. The defender processes an input in multiple channels. Each channel introduces its own randomization in a special transform domain based on a secret key shared between the training and testing stages. Such a transform based randomization with a shared key preserves the gradients in key-defined sub-spaces for the defender but it prevents gradient back propagation and the creation of various bypass systems for the attacker. An additional benefit of multi-channel randomization is the aggregation that fuses soft-outputs from all channels, thus increasing the reliability of the final score. The sharing of a secret key creates an information advantage to the defender. Experimental evaluation demonstrates an increased robustness of the proposed method to a number of known state-of-the-art attacks.

In this paper, we address the problem of data reconstruction from privacy-protected templates, based on recent concept of sparse ternary coding with ambiguization (STCA). The STCA is a generalization of randomization techniques which includes random projections, lossy quantization, and addition of ambiguization noise to satisfy the privacy-utility trade-off requirements. The theoretical privacy-preserving properties of STCA have been validated on synthetic data. However, the applicability of STCA to real data and potential threats linked to reconstruction based on recent deep reconstruction algorithms are still open problems. Our results demonstrate that STCA still achieves the claimed theoretical performance when facing deep reconstruction attacks for the synthetic i.i.d. data, while for real images special measures are required to guarantee proper protection of the templates.

In this paper, we address a problem of machine learning system vulnerability to adversarial attacks. We propose and investigate a Key based Diversified Aggregation (KDA) mechanism as a defense strategy. The KDA assumes that the attacker (i) knows the architecture of classifier and the used de-fense strategy, (ii) has an access to the training data set but (iii) does not know the secret key. The robustness of the system is achieved by a specially designed key based randomization. The proposed randomization prevents the gradients' back propagation or the creating of a "bypass" system. The randomization is performed simultaneously in several channels and a multi-channel aggregation stabilizes the results of randomization by aggregating soft outputs from each classifier in multi-channel system. The performed experimental evaluation demonstrates a high robustness and universality of the KDA against the most efficient gradient based attacks like those proposed by N. Carlini and D. Wagner [1] and the non-gradient based sparse adversarial perturbations like OnePixel attacks [2].

This paper addresses the problem of Approximate Nearest Neighbor (ANN) search in pattern recognition where feature vectors in a database are encoded as compact codes in order to speed-up the similarity search in large-scale databases. Considering the ANN problem from an information-theoretic perspective, we interpret it as an encoding, which maps the original feature vectors to a less entropic sparse representation while requiring them to be as informative as possible. We then define the coding gain for ANN search using information-theoretic measures. We next show that the classical approach to this problem, which consists of binarization of the projected vectors is sub-optimal. Instead, a properly designed ternary encoding achieves higher coding gains and lower complexity.

In this paper, we compare two methods that can be used by the anti-counterfeiting industry to protect physical objects, which are either based on an object's natural randomness or on artificial randomness embedded on the object. We show that the considered verification architectures rely either on a comparison between an enrolled fingerprint and an extracted one or between a tag and a fingerprint. We compare these setups from detection-theoretic perspectives for both types of architectures. Authentication performance using false and miss error probabilities of the two systems are analysed and then compared using two practical setups. We highlight the advantages and limitations of each architecture. These theoretical results derived for binary fingerprints are useful to construct and optimise practical methods and to help select the appropriate architecture.

This papers presents an analysis on Active Content Fingerprint (aCFP) for local (patch based) image descriptors. A generalization is proposed, the reduction of the aCFP with linear modulation to a constrained projection problem is shown and the optimal solution is given. The constrained projection problem addresses the linear modulation by a constraint on the properties of the resulting local descriptor. A computer simulation using local image patches, extracted from publicly available data sets is provided, demonstrating the advantages under several signal processing distortions.

This paper presents solutions to the local patch based Active Content Fingerprint (aCFP) with linear modulation, general linear feature map and convex constraints on the properties of the local feature descriptor. A direct approximation of the linear feature map such that the image distortion is as small as possible and the approximate linear feature map is as close as possible to the original map is proposed. Then an explicit regularization of the trade-off between the modulation distortion and the robustness of the local feature is introduced trough a novel problem formulation. A computer simulation using local image patches, extracted from publicly available data set is provided, demonstrating the advantages under: additive white Gaussian noise (AWGN), lossy JPEG compression and projective geometrical transform distortions.

Content identification based on digital content fingerprinting attracts significant attention in different emerging applications. In this paper, we consider content identification based on the sign-magnitude decomposition of fingerprint codewords and analyze the achievable rates for sign and magnitude components. We demonstrate that the bit robustness in the sign channel, often used in binary fingerprinting, is determined by the value of the corresponding magnitude component. Correspondingly, one can distinguish between two systems depending how the information about the magnitude component is used at the decoding process, i.e., hard fingerprinting when this information is disregarded, and soft fingerprinting when this information is used. To reveal the advantages of soft information at the decoding, we consider a case of soft fingerprinting where the decoder has access to the complete information about the uncoded magnitude component. However, since it requires a lot of extra memory storage or secure communication, the magnitude information is often quantized to a single bit or extracted directly from the noisy observation. To generalize the existing methods and estimate the impact of quantization and noise in the side information about the magnitude components on the achievable rate, we introduce a channel splitting approach and reveal certain interesting phenomena related to channel polarization. We demonstrate that under proper quantization of the magnitude component, one can clearly observe the existence of strong components whose sign is very robust, even to strong distortions. We demonstrate that under certain conditions, a great portion of the rate in the sign channel is concentrated in strong channel components. Finally, we demonstrate how to use the channel splitting property in the design of efficient low-complexity identification methods.

Many state-of-the-art methods in image retrieval, classification and copy detection are based on the Bag-of-Features (BOF) framework. However, the performance of these systems is mostly experimentally evaluated and little results are reported on theoretical performance. In this paper, we present a statistical framework that makes it possible to analyse the performance of a simple BOF-system and to better understand the impact of different design elements such as the robustness of descriptors, the accuracy of encoding/assignment, information preserving pooling and finally decision making. The proposed framework can be also of interest for a security and privacy analysis of BOF systems.

In this paper, we extend a new framework introduced as active content fingerprinting in [1] 1 that takes the best from the two worlds of content fingerprinting and digital watermarking to overcome some of the fundamental restrictions of these techniques in terms of performance and complexity. In the proposed framework, contents are modified in a way similar to watermarking to extract more robust fingerprints in contrast to conventional content fingerprinting. We investigate the performance of two modulation techniques based on unidimensional shrinkage and multidimensional lattice quantization. The simulation results on real images demonstrate the high efficiency of the proposed methods facing low-quality compression and additive noise.

In this paper we advocate a new technique for the fast identification of physical objects based on their physical unclonable features (surface microstructures). The proposed identification method is based on soft fingerprinting and consists of two stages: at the first stage the list of possible candidates is estimated based on the most reliable bits of a soft fingerprint and the traditional maximum likelihood decoding is applied to the obtained list to find a single best match at the second stage. The soft fingerprint is computed based on random projections with a sign-magnitude decomposition of projected coefficients. The estimate of a bit reliability is deduced directly from the observed coefficients. We investigate different decoding strategies to estimate the list of candidates, which minimize the probability of miss of the right index on the list. The obtained results show the flexibility of the proposed identification method to provide the performance-complexity trade-off.

This paper is dedicated to the analysis of object identification under desynchronization distortions. While this class of degradations is almost unavoidable in the functionality of such systems especially at the verification stage via acquisition imperfections, currently available their performance limits do not take its impact into consideration. In this paper we will try to close this gap providing the estimation of the achievable identification rates due to desynchronization distortions. Finally, the impact of the codeword length on the identification performance is justified.

Content identification based on digital fingerprinting attracts a lot of attention in different emerging applications. In this paper, we consider digital identification based on the sign-magnitude decomposition of fingerprint codewords and analyze the achievable rates for each component. We introduce a channel splitting approach and reveal certain interesting phenomena related to channel polarization. It is demonstrated that under certain conditions almost all rate in the sign channel is concentrated in reliable components, this can be of interest for complexity and security in various content identification applications. The envisioned extensions cover applications where the input and output alphabets of the channel are different at the encoding and decoding stages. Additionally, the reduction of the input data dimensionality at the encoding/enrollment stage can increase the cryptographic protection in terms of privacy leakage and simplify the decoding algorithms in biometric applications.

In recent years, content identification based on digital fingerprinting attracts a lot of attention in different emerging applications. At the same time, the theoretical analysis of digital fingerprinting systems for finite length case remains an open issue. Additionally, privacy leaks caused by fingerprint storage, distribution and sharing in a public domain via third party outsourced services cause certain concerns in the cryptographic community. In this paper, we perform an information-theoretic analysis of finite length digital fingerprinting systems in a private content identification setup and reveal certain connections between fingerprint based content identification and Forney's erasure/list decoding. Along this analysis, we also consider complexity issues of fast content identification in large databases on remote untrusted servers.

In light of the recent development of multimedia and networking technologies, an exponentially increasing amount of content is available via various public services. That is why content identification attracts a lot of attention. One possible technology for content identification is based on digital fingerprinting. When trying to establish information-theoretic limits in this application, usually it is assumed that the codewords are of infinite length and that a jointly typical decoder is used in the analysis. These assumptions represent a certain over-generalization for the majority of practical applications. Consequently, the impact of the finite length on the mentioned limits remains an open and largely unexplored problem. Furthermore, leaking of privacy-related information to third parties due to storage, distribution and sharing of fingerprinting data represents an emerging research issue that should be addressed carefully. This paper contains an information-theoretic analysis of finite length digital fingerprinting under privacy constraints. A particular link between the considered setup and Forney's erasure/list decoding [1] is presented. Finally, complexity issues of reliable identification in large databases are addressed.