SimMIM is a widely used method for pretraining vision transformers using masked image modeling. However, despite its success in fine-tuning performance, it has been shown to perform sub-optimally when used for linear probing. We propose an efficient patch-wise weighting derived from keypoint features which captures the local information and provides better context during SimMIM’s reconstruction phase. Our method, KAMIM, improves the top-1 linear probing accuracy from 16.12% to 33.97%, and finetuning accuracy from 76.78% to 77.3% when tested on ImageNet-1K dataset with a ViT-B trained for the same number of epochs while taking the same wall-clock-time. We conduct extensive testing on different datasets, keypoint extractors, and model architectures and observe that patch-wise weighting augments linear probing performance for larger pretraining datasets. We also analyze the learned representations of a ViT-B trained using KAMIM and observe that they behave similar to contrastive learning with longer attention distances. Our code is available at https://github.com/madhava20217/KAMIM.

Despite the rapid advancement in the field of image recognition, the processing of high-resolution imagery remains a computational challenge. However, this processing is pivotal for extracting detailed object insights in areas ranging from autonomous vehicle navigation to medical imaging analyses. Our study introduces a framework aimed at mitigating these challenges by leveraging memory efficient patch based processing for high resolution images. It incorporates a global context representation alongside local patch information, enabling a comprehensive understanding of the image content. In contrast to traditional training methods which are limited by memory constraints, our method enables training of ultra high resolution images. We demonstrate the effectiveness of our method through superior performance on 4 different benchmarks across classification, object detection, and segmentation. Notably, the proposed method achieves strong performance even on resource-constrained devices like Jetson Nano. Our code is available at https://shorturl.at/ikVib.

Optimal transport (OT) based adversarial robustness has received some traction in the recent past. Though it is at a nascent stage, OT has a sound potential in robustifying deep learning models. Interestingly, OT barycenters demonstrate a good robustness against adversarial attacks. Owing to the computationally expensive nature of OT barycenters, they have not been investigated towards adversarial robustness. In this work, we propose a new barycenter, namely Beckman barycenter, which can be computed efficiently and used for training the network to defend against adversarial attacks in conjunction with adversarial training. We propose a novel formulation of Beckman barycenter and analytically obtain the barycenter using the marginals of the input image. We show that the Beckman barycenter can be used to train adversarially trained networks to improve the robustness. Our training is extremely efficient as it requires only a single epoch of fine-tuning. Elaborate experiments on CIFAR-10, CIFAR-100 and Tiny ImageNet demonstrate that training an adversarially robust network with Beckman barycenter can significantly increase the performance. Under auto attack, we get a maximum boost of 10% in CIFAR-10, 8.34% in CIFAR-100 and 11.51% in Tiny ImageNet. We further show that our method outperforms state-of-art adversarial purification methods. Code is available at: https://github.com/Visual-Conception-Group/test-barycentric-defense.

Adversarial purification using generative models demonstrates strong adversarial defense performance. These methods are classifier and attack-agnostic, making them versatile but often computationally intensive. Recent strides in diffusion and score networks have improved image generation and, by extension, adversarial purification. Another highly efficient class of adversarial defense methods known as adversarial training requires specific knowledge of attack vectors, forcing them to be trained extensively on adversarial examples. To overcome these limitations, we introduce a new framework, namely Language Guided Adversarial Purification (LGAP), utilizing pre-trained diffusion models and caption generators to defend against adversarial attacks. Given an input image, our method first generates a caption, which is then used to guide the adversarial purification process through a diffusion network. Our approach has been evaluated against strong adversarial attacks, proving its effectiveness in enhancing adversarial robustness. Our results indicate that LGAP outperforms most existing adversarial defense techniques without requiring specialized network training. This underscores the generalizability of models trained on large datasets, highlighting a promising direction for further research. Our code is available at github.com/Visual-Conception-Group/LGAP.

Adversarial attacks have gained significant attention in object re-identification (Re-Id). However, very few works target defense, and they primarily adopt adversarial training. While adversarial training has been shown to be a major line of defense, we observe that vanilla adversarial training alone does not provide a robust defense against adversarial attacks. Towards this, we propose a novel meta perturbed defense algorithm for Re-Id task. Our contributions are, (i) we introduce anisotropic and isotropic perturbations to design a stochastic neural network, and train it with a novel meta-learning strategy with tasks as vanilla, perturbed, and perturbed-adversarial training; (ii) we show the generalizability of our model against various unseen attacks; and (iii) we derive a novel feature covariance alignment (FCA) loss which gives us a high clean performance while providing robustness against different attacks. Extensive experiments on Market-1501, MSMT17 and VeRi-776 reveal SOTA performance.1

Adversarial attacks have been recently investigated in person re-identification. These attacks perform well under cross dataset or cross model setting. However, the challenges present in cross-dataset cross-model scenario does not allow these models to achieve similar accuracy. To this end, we propose our method with the goal of achieving better transferability against different models and across datasets. We generate a mask to obtain better performance across models and use meta learning to boost the generalizability in the challenging cross-dataset cross-model setting. Experiments on Market-1501, DukeMTMC-reID and MSMT-17 demonstrate favorable results compared to other attacks.

Unsupervised domain adaptation for person re-identification (Re-ID) suffers severe domain discrepancies between source and target domains. To reduce the domain shift caused by the changes of context, camera style, or viewpoint, existing methods in this field fine-tune and adapt the Re-ID model with augmented samples, either translating source samples to the target style or assigning pseudo labels to the target. The former methods may lose identity details but keep redundant source background during translation. In contrast, the latter techniques may give noisy labels when the model meets the unseen background and person pose. We mitigate the domain shift in the former translation direction by cyclically decoupling environment and identity-related features. We propose a novel individual-preserving and environmental-switching cyclic generation network (IPES-GAN). Our network has the following distinct features: 1)Decoupled features instead of fused features:we encode the images into an individual part and an environmental part, which are proved beneficial to generation and adaptation; 2)Cyclic generation instead of one-step adaptive generation. We swap source and target environment features to generate cross-domain images with preserved identity-related features conditioned with source (target) background features and then changed again to generate back the input image so that cyclic generation runs in a self-supervised way. Experiments carried out on two significant benchmarks: Market-1501 and DukeMTMC-Reid, reveal state-of-the-art performance.

Wasserstein metric based adversarial attacks have attracted a great interest in the recent past. Even though they exhibit strong attacks, surprisingly, they have not been investigated for defense. In this work, we demonstrate that barycenters computed in Wasserstein space can act as a measure of defense against adversarial attacks. We compute the barycenter using marginals obtained from the given image and demonstrate its effectiveness in defense even without any adversarial training. We further analyse the barycenters using GradCam to understand their defensive characteristics. Elaborate experiments on MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100 demonstrate a significant increase in the robustness of victim classifiers. Our code is available at https://github.com/Visual-Conception-Group/Barycentric-Defense.

Correlation Filters (CF) are a popular choice for visual object tracking due to their efficiency in the frequency domain. Convolutional and hand-crafted features are jointly used when learning a filter, however, these features are not uniformly important when tracking a target. Given this observation, spatial and temporal regularization and attention models have been investigated. However, these models do not consider the interaction between different feature channels. As a result, dissimilar weights are assigned to similar feature channels. To address this issue, we propose a channel attention model and study two different regularization methods for attention. We investigate the application of channel regularization to emphasize important feature channels; and graph regularization which increases the likelihood of similar feature channels obtaining similar weights. The proposed formulation can be efficiently solved via the alternating direction method of multipliers. We first show the advantages of using the proposed channel regularization by demonstrating its performance when applied to two existing CF trackers. This is followed by analyzing the effect of using the proposed channel-graph regularization for CF based tracking. The evaluation is performed on publicly available tracking datasets: OTB100, TC128, VOT-2017, VOT-2019, LaSOT, UAV123, and GOT-10k. Evaluation over multiple challenges and a comparative analysis with existing top-ranked trackers shows that our formulation improves the discriminative power of the learned CF, preventing tracker drift during challenging scenarios.

space. In this work, we propose a novel approach for generating adversarial samples using Wasserstein distance. Unlike previous approaches, we use an unbalanced optimal transport formulation which is naturally suited for images. We first compute an adversarial sample using a gradient step and then project the resultant image into Wasserstein ball with respect to original sample. The attack introduces perturbation in the form of pixel mass distribution which is guided by a cost metric. Elaborate experiments on MNIST, Fashion-MNIST, CIFAR-10 and Tiny ImageNet demonstrate a sharp decrease in the performance of state-of-art classifiers. We also perform experiments with adversarially trained classifiers and show that our system achieves superior performance in terms of adversarial defense against several state-of-art attacks. Our code and pre-trained models are available at https://bit.ly/2SQBR4E.

Remote sensing through satellite imagery is applied widely for environmental control, urban planning and land cover classification. To this end, supervised deep learning models can fully exploit the potential of satellite images. However, such models require large annotated datasets. Therefore, through this study we present two novel labelled satellite image datasets, namely In-Sat1, based on Sentinel-2 Satellite Imagery and Google Earth Imagery covering Indian subcontinent regions. We provide benchmarks and detailed analysis for these datasets using state-of-the-art and our improved remote sensing deep learning models which achieve 91% and 81% overall accuracy in region-wise split setting and 98% and 94% overall accuracy in class-wise split setting for Sentinel-2 data and Google Earth data respectively. We also demonstrate the application of our system for change detection.

Unsupervised cross-domain Person Re-Identification (Re-ID) suffers from severe domain gap issue. While different works address this issue, bridging domain gap with high-level representation is hard as it comprises of entangled information including identity, background, occlusion, and other domain-specific variations. In this paper, we propose a disentangled reconstruction method to address the domain-shift problem for Re-ID in an unsupervised manner. To this end, we have two major contributions. First, we propose to disentangle identity-relevant and identity-irrelevant features from person images. Second, in the target domain, we explicitly consider the camera style transfer images as a data augmentation to address intra-domain discrepancy and to learn the camera invariant features. Experimental results on the challenging benchmarks of Market-1501 and DukeMTMC-reID demonstrate that our proposed method achieves competitive performance.

This paper proposes a novel Importance Guided Sparse Spatio-Temporal Regularization based Correlation Filter (IGSSTRCF) tracker. Our formulation explicitly models the variations in the correlation filters and associated spatial weights in successive frames. By imposing a sparsity penalty on these variations, the formulation ensures that only relevant changes are incorporated during updates. This results in more robust filter coefficients that minimize the tracking drift. The IGSSTRCF also includes an adaptive channel importance estimation strategy that assigns an importance weight to each feature channel during training. The proposed formulation is efficiently solved via the alternating direction method of multipliers. A comparative analysis is shown on TC128, UAV123, VOT-2017, and VOT-2019 datasets; and we present an ablation study to demonstrate the contribution of each component of the IGSSTRCF. It is observed that we outperform several state-of-the-art trackers and each component of the proposed IGSSTRCF contributes positively towards tracker performance.

Visible-infrared person re-identification (RGB-IR ReID) is extremely important for the surveillance applications under poor illumination conditions. Since the difference in the feature representations not only lies in the person' pose, viewpoint or illumination variations, but also comes from huge spectrum discrepancy, the task becomes practically very challenging. Existing RGB-IR ReID models focus on bridging the gap between RGB and IR images through shared feature embedding, subspace learning or via adversarial learning. However, these methods do not explicitly disregard the spectrum information which is otherwise irrelevant for ReID. Further, adversarial learning methods has less promising convergence. This motivates us to design a non-adversarial and fast disentanglement method to disentangle the spectrum information while learning the identity discriminative features. To extract these features, we propose a novel network with disentanglement loss which can distill identity features and dispel spectrum features. Our network has two branches, spectrum dispelling and spectrum distilling branch. On spectrum dispelling branch, we apply identification loss to learn the identity related and spectrum disentangled features. On spectrum distilling branch, we apply an identity-dispeller loss to fool the identity classifier so that it primarily learns spectrum related information. The entire network is trained in an end-to-end manner, which minimizes spectrum information and maximizes invariant identity relevant information at spectrum dispelling branch. Extensive experiments on existing datasets demonstrate the superior performance of our approach compared to the state-of-the-art.

Video acquisition using dashboard-mounted cameras has recently achieved massive popularity around the world. One of the major developments following the dash-cam's popularity is that videos captured by them can be used as testimony during scenarios, like traffic violations and accidents. The widespread deployment of dash-cams brings new problems ranging from the compromise of privacy by uploading these videos on public websites using videos captured from other cars for making fraudulent claims. Therefore, there is a compelling need to address the problems associated with the usage of dash-cam videos. In this paper, we discuss and highlight the importance of the emerging area of multimedia vehicle forensics. We propose an algorithm for linking a dash-cam video to a specific car. The proposed algorithm is useful for various applications, for example, insurance companies can authenticate the origin of video before processing the claim. In a different scenario of illegitimate video upload on the Web, the video can be traced back to the car it originated from. To this end, we make use of motion blur extracted from dash-cam videos for generating a discriminative feature. We observe that the subtle motion pattern of every vehicle can serve as its unique signature. We extract motion blur from dash-cam videos and use random forest trees for classifying the vehicle correctly. The experimental results on thousands of frames obtained from dash-cam videos of several cars show the effectiveness of our approach. We further investigate the process of forging the signature of a car and propose a counter forensics method to detect such forgery. Also, we discuss the application of our technique to other potential platforms where the camera can be mounted, for example, on the chest of a person. We believe that ours is the first work that describes this new area of research.

Road safety mapping using satellite images is a cost-effective but a challenging problem for smart city planning. The scarcity of labeled data, misalignment and ambiguity makes it hard to learn efficient embeddings in order to classify between safe and dangerous road segments. In this paper, we address the challenges using a region guided attention network. In our model, we extract global features from a base network and augment it with local features obtained using the region guided attention network. In addition, we perform domain adaptation for unlabeled target data. In order to bridge the gap between safe samples and dangerous samples from source and target respectively, we propose a loss function based on within and between class covariance matrices. We conduct experiments on a public dataset of London to show that the algorithm achieves significant results with the classification accuracy of 86.21%. We obtain an increase of 4% accuracy for NYC using domain adaptation network.

Metric learning is one of the fundamental problems in person re-identification. However, a good number of the current techniques do not generalize well in the presence of outliers. Toward this, we present a robust discriminative subspace learning technique in this letter. We learn the subspace by maximizing the ratio of between class covariance and within class covariance using L1 norm instead of the conventional L2 norm. We theoretically show the iterative approach of computing the subspace. In case of noisy data, our experimental results demonstrate an overall average improvement of more than 4.2% in Rank-1 accuracy on CUHK03, Market1501, and DukeMTMC4ReID compared to popular metric learning algorithms.

Video based person re-identification has received significant attention in the recent past due to advancement in deep learning techniques. However, it is still a challenging problem because of inherent dynamic nature of videos. Additionally, lack of sufficient annotated dataset may lead to overfitting issues while training deep networks. In this paper, we propose a spatio-temporal transfer learning approach using 3D-CNN for video based person re-identification. To address the issue of insufficient labelled data and transfer the knowledge, we use a pre-trained 3D-CNN model of Sports-1M dataset and perform fine-tuning on multiple domain datasets such as PRID-2011, iLIDS-VID, MARS and an aerial video dataset simultaneously. Learning features from multiple domain data is of significant value because of large variation which otherwise is not possible to obtain from small individual datasets. In our experiments, we show that the fine-tuned transferred features encode robust representations and enhance the re-identification accuracy. Further, to boost the performance, we apply XQDA metric learning. Experiments conducted on all the four datasets show that the proposed framework outperforms the popular methods by an average improvement of 4% or more. In addition, we analyse the network's robustness against adversarial examples and show that the proposed 3D-CNN network has better resilience compared to 2D-CNN used in most of the existing algorithms.

Visual object tracking involves the challenging task of scale adaptation to the changing object appearance. Sometimes, this leads to excessive expansion or contraction of the estimated bounding box. Towards this, several generative, discriminatory and hybrid models have been proposed. In this paper, we propose to apply Backward Validation Tracking (BVT) along with an online SVM. BVT has an advantage that it creates a model pool depending on the amount of variation of object's appearance in subsequent frames. Thus while tracking an object, not only is the current appearance taken into account, but so are the previous appearances which are stored in the model pool. Further, in order to improve on the appearance model adaptation, we use an online SVM. The online SVM is a discriminatory algorithm which allows us to distinguish between foreground and background objects. On account of its online nature, the SVM adapts to the updates in the appearance of the target object. We perform extensive experiments on the Object Tracking Benchmark (OTB) dataset. Experimental results prove that the proposed tracker outperforms several popular trackers, in terms of both overlap ratio and precision.

The use of dashboard-mounted video cameras is rapidly spreading in many countries around the world. Widespread usage of dash-cams brings new problems, for example, dash-cam videos are uploaded on public websites which contain footage of other cars with the number-plates visible. This can potentially compromise privacy. Further, dash-cam videos can be used as evidence in case of accidents. There have been even cases of usage of dash-cam videos for insurance claims. Not only genuine claims can be made but fraudulent claims using some other cars footage can be used. The use as well as misuse of dash-cam videos is going to be widely prevalent in the near future. In this paper, we present a solution to problem of identifying the vehicle in which the dashboard camera is mounted. Our technique can be used by insurance companies for authenticating the source of origin (the vehicle on which the camera is mounted) of video before processing the insurance claim. We make use of features extracted from motion blur as a feature which is generated due to the unique motion of a vehicle. We have found that the subtle motion pattern of every vehicle acts as unique signature. To the best of our knowledge, ours is the first work that describes this new area of research.

The data generated on social media sites continues to grow at an increasing rate with more than 36% of tweets containing images making the dominance of multimedia content evidently visible. This massive user generated content has become a reflection of world events. In order to enhance the ability and effectiveness to consume this plethora of data, summarization of these events is needed. However, very few studies have exploited the images attached with social media events to summarize them using "mid-level visual elements". These are the entities which are both representative and discriminative to the target dataset besides being human-readable and hence more informative.

Hypertension (high blood pressure) is the leading cause for increasing number of premature deaths due to cardiovascular diseases. Continuous hypertension screening seems to be a promising approach in order to take appropriate steps to alleviate hypertension-related diseases. Many studies have shown that physiological signal like Photoplethysmogram (PPG) can be reliably used for predicting the Blood Pressure (BP) and Heart Rate (HR). However, the existing approaches use a transmission or reflective type wearable sensor to collect the PPG signal. These sensors are bulky and mostly require an assistance of a trained medical practitioner; which preclude these approaches from continuous BP monitoring outside the medical centers. In this paper, we propose a novel touchless approach that predicts BP and HR using the face video based PPG. Since the facial video can easily be captured using a consumer grade camera, this approach is a convenient way for continuous hypertension monitoring outside the medical centers. The approach is validated using the face video data collected in our lab, with the ground truth BP and HR measured using a clinically approved BP monitor OMRON HBP1300. Accuracy of the method is measured in terms of normalized mean square error, mean absolute error and error standard deviation; which complies with the standards mentioned by Association for the Advancement of Medical Instrumentation. Two-tailed dependent sample t-test is also conducted to verify that there is no statistically significant difference between the BP and HR predicted using the proposed approach and the BP and HR measured using OMRON.

Photo response nonuniformity (PRNU) is consider as reliable camera fingerprint for identifying source of a digital images. Digital cameras use various image processing operations to map linear color measurements (raw data) into nonlinear narrow gamut image. This nonlinear transformation affects estimation of PRNU. To undo the effect of nonlinear transformation, in this letter, we propose to estimate PRNU from probabilistically obtained raw values. Since not all cameras provide raw values as their output, we propose to compute estimate of raw values from the JPEG images using probabilistic color derendering procedure. The estimated raw values are modeled as a Poisson process and then maximum likelihood estimation (MLE) is used for PRNU estimation. The experimental results show that, the digital camera identification using our proposed PRNU estimate is better than using other popular PRNU estimate.

Detecting Contrast Enhancement (CE) in images and anti-forensic approaches against such detectors have gained much attention in multimedia forensics lately. Several contrast enhancement detectors analyze the first order statistics such as gray-level histogram of images to determine whether an image is CE or not. In order to counter these detectors various anti-forensic techniques have been proposed. This led to a technique that utilized second order statistics of images for CE detection. In this letter, we propose an effective anti-forensic approach that performs CE without significant distortion in both the first and second order statistics of the enhanced image. We formulate an optimization problem using a variant of the well known Total Variation (TV) norm image restoration formulation. Experiments show that the algorithm effectively overcomes the first and second order statistics based detectors without loss in quality of the enhanced image.

The availability of intelligent image editing techniques and antiforensic algorithms, make it convenient to manipulate an image and to hide the artifacts that it might have produced in the process. Real world forgeries are generally followed by the application of enhancement techniques such as filtering and/or conversion of the image format to suppress the forgery artifacts. Though several techniques evolved in the direction of detecting some of these manipulations, additional operations like recompression, nonlinear filtering, and other antiforensic methods during forgery are not deeply investigated. Toward this, we propose a robust method to detect whether a given image has undergone filtering (linear or nonlinear) based enhancement, possibly followed by format conversion after forgery. In the proposed method, JPEG quantization noise is obtained using natural image prior and quantization noise models. Transition probability features extracted from the quantization noise are used for machine learning based detection and classification. We test the effectiveness of the algorithm in classifying the class of the filter applied and the efficacy in detecting filtering in low resolution images. Experiments are performed to compare the performance of the proposed technique with state-of-the-art forensic filtering detection algorithms. It is found that the proposed technique is superior in most of the cases. Also, experiments against popular antiforensic algorithms show the counter antiforensic robustness of the proposed technique.

Smart image editing and processing techniques make it easier to manipulate an image convincingly and also hide any artifacts of tampering. Common real world forgeries can be accompanied by enhancement operations like filtering, compression and/or format conversion to suppress forgery artifacts. Out of these enhancement operations, filtering is very common and has received a lot of attention in forensics research lately. However, different filtering operations and image formats are not investigated deeply and simultaneously. We propose an algorithm to detect if a given image has undergone filtering based enhancement irrespective of the format of image or the type of filter applied. In the proposed algorithm, we exploit the correlation of spatial domain quantization noise of an image by extracting transition probability features and classify the image as filtered or unfiltered. Experiments are performed to evaluate the robustness and compare the performance of the proposed technique with popular forensic filtering detection algorithms and is found to be superior in most of the cases.

Intelligent video editing techniques can be used to tamper videos such as surveillance camera videos, defeating their potential to be used as evidence in a court of law. In this paper, we propose a technique to detect forgery in MPEG videos by analyzing the frame's compression noise characteristics. The compression noise is extracted from spatial domain by using a modified Huber Markov Random Field (HMRF) as a prior for image. The transition probability matrices of the extracted noise are used as features to classify a given video as single compressed or double compressed. The experiment is conducted on different YUV sequences with different scale factors. The efficiency of our classification is observed to be higher relative to the state of the art detection algorithms.

In this paper, we propose a novel technique to detect double quantization, which results due to double compression of a tampered video. The proposed algorithm uses principles of estimation theory to detect double quantization. Each pixel of a given frame is estimated from the spatially colocated pixels of all the other frames in a Group of Picture (GOP). The error between the true and estimated value is subjected to a threshold to identify the double compressed frame or frames in a GOP. The advantage of this algorithm is that it can detect tampering of I, P or B frames in a GOP with high accuracy. In addition, the technique can also detect forgery under wide range of double compression bitrates or quantization scale factors. We compare our experimental results against popular video forgery detection techniques and establish the effectiveness of the proposed technique.

This paper analyses an image watermarking scheme based on Support Vector Regression (SVR) proposed by R. Shen et al. We describe various attacks against this scheme and show that watermark tampering can be done even when one does not know the secret key used to embed the watermark. Next we discuss methods to extract the keys used in the scheme under various usage scenarios. Our results show that Shen et al.'s scheme is not secure.

In this paper, we propose a novel video forgery detection technique to detect the spatial and temporal copy paste tampering. It is a challenge to detect the spatial and temporal copy-paste tampering in videos as the forged patch may drastically vary in terms of size, compression rate and compression type (I, B or P) or other changes such as scaling and filtering. In our proposed algorithm, the copy-paste forgery detection is based on Histogram of Oriented Gradients (HOG) feature matching and video compression properties. The benefit of using HOG features is that they are robust against various signal processing manipulations. The experimental results show that the forgery detection performance is very effective. We also compare our results against a popular copy-paste forgery detection algorithm. In addition, we analyze the experimental results for different forged patch sizes under varying degree of modifications such as compression, scaling and filtering.

In Digital Asset Management systems, media is often handled in compressed and encrypted form. In this paper, we propose a novel partially compressed-encrypted robust MP3 audio watermarking technique. However, arbitrary embedding of a watermark in a partially compressed encrypted MP3 audio can cause drastic degradation of the quality as the underlying change may result in random decrypted values. In addition, encryption may result in very low compression efficiency. Thus the challenge is to design a watermarking technique that provides good watermarked audio quality and at the same time gives good compression efficiency. While the proposed technique embeds watermark in the partially compressed-encrypted domain, the extraction of watermark can be done in the encrypted or decrypted domains. The experiments show that the watermarked audio quality is good and the reduction in compression efficiency is low. The experimental results also show that the proposed watermarking technique is robust to common signal processing attacks.

Digital asset management systems (DAMS) generally handle media data in a compressed and encrypted form. It is sometimes necessary to watermark these compressed encrypted media items in the compressed-encrypted domain itself for tamper detection or ownership declaration or copyright management purposes. It is a challenge to watermark these compressed encrypted streams as the compression process would have packed the information of raw media into a low number of bits and encryption would have randomized the compressed bit stream. Attempting to watermark such a randomized bit stream can cause a dramatic degradation of the media quality. Thus it is necessary to choose an encryption scheme that is both secure and will allow watermarking in a predictable manner in the compressed encrypted domain. In this paper, we propose a robust watermarking algorithm to watermark JPEG2000 compressed and encrypted images. The encryption algorithm we propose to use is a stream cipher. While the proposed technique embeds watermark in the compressed-encrypted domain, the extraction of watermark can be done in the decrypted domain. We investigate in detail the embedding capacity, robustness, perceptual quality and security of the proposed algorithm, using these watermarking schemes: Spread Spectrum (SS), Scalar Costa Scheme Quantization Index Modulation (SCS-QIM), and Rational Dither Modulation (RDM).

In digital rights management (DRM) systems, digital media is often distributed by multiple levels of distributors in a compressed and encrypted format. The distributors in the chain face the problem of embedding their watermark in compressed, encrypted domain for copyright violation detection purpose. In this paper, we propose a robust watermark embedding technique for JPEG2000 compressed and encrypted images. While the proposed technique embeds watermark in the compressed-encrypted domain, the extraction of watermark can be done either in decrypted domain or in encrypted domain.

Multiparty multilevel digital rights management (DRM) architecture involving several levels of distributors in between an owner and a consumer has been suggested as an alternative business model to the traditional two-party (buyer-seller) DRM architecture for digital content delivery. In the two-party DRM architecture, cryptographic techniques are used for secure delivery of the content, and watermarking techniques are used for protecting the rights of the seller and the buyer. The cryptographic protocols used in the two-party case for secure content delivery can be directly applied to the multiparty multilevel case. However, the watermarking protocols used in the two-party case may not directly carry over to the multiparty multilevel case, as it needs to address the simultaneous security concerns of multiple parties such as the owner, multiple levels of distributors, and consumers. Towards this, in this paper, we propose a joint digital watermarking scheme using Chinese remainder theorem for the multiparty multilevel DRM architecture. In the proposed scheme, watermark information is jointly created by all the parties involved; then a watermark signal is generated out of it and embedded into the content. This scheme takes care of the security concerns of all parties involved. Further, in the event of finding an illegal copy of the content, the violator(s) can be traced back.