State-of-the-art (SOTA) fact-checking systems combat misinformation by employing autonomous LLM-based agents to decompose complex claims into smaller sub-claims, verify each sub-claim individually, and aggregate the partial results to produce verdicts with justifications (explanations for the verdicts). The security of these systems is crucial, as compromised fact-checkers can amplify misinformation, but remains largely underexplored. To bridge this gap, this work introduces a novel threat model against such fact-checking systems and presents Fact2Fiction, the first poisoning attack framework targeting SOTA agentic fact-checking systems. Fact2Fiction employs LLMs to mimic the decomposition strategy and exploit system-generated justifications to craft tailored malicious evidences that compromise sub-claim verification. Extensive experiments demonstrate that Fact2Fiction achieves 8.9%-21.2% higher attack success rates than SOTA attacks across various poisoning budgets and exposes security weaknesses in existing fact-checking systems, highlighting the need for defensive countermeasures.

In domains like driving and healthcare, deep learning models often rely on large, diverse datasets that can inadvertently harbor backdoor attacks. In this paper, we propose ABDP, a novel post-processing defense method, to effectively remove backdoor contamination from datasets and generate clean models without relying on any pre-existing clean data. ABDP capitalizes on the intrinsic link between untargeted adversarial attacks and backdoor attacks to detect the presence of backdoor attacks within trained models and ascertain their target labels. Subsequently, it trains a clean model capable of recognizing all labels except the target label, thus treating poisoned data as in-distribution and clean data of the target label as out-of-distribution. This distinction enables the identification of backdoor poisoned data. Finally, ABDP applies unlearning techniques to effectively eradicate the backdoor from the model. Extensive experimental evaluations across diverse datasets and against multiple backdoor attack scenarios validate the robustness and state-of-the-art performance of our approach. By employing ABDP for data and model cleansing, the attack success rate of resulting models is reduced to 1% or less, while retaining approximately 70% or more clean data at a true positive rate of 0.01 false positive rate. Notably, ABDP exhibits no adverse impact when applied to purely clean datasets, owing to its ability to detect backdoor presence in models before cleansing. Thus, our proposed method achieves state-of-the-art performance in cleansing both backdoor-poisoned data and backdoor models. The code for ABDP will be made available upon publication of the paper.

This survey examines evaluation methods for large language model (LLM)-based agents in multi-turn conversational settings. Using a PRISMA-inspired framework, we systematically reviewed nearly 250 scholarly sources, capturing the state-of-the-art from various venues of publication, and establishing a solid foundation for our analysis. Our study offers a structured approach by developing two interrelated taxonomy systems: one that defines what to evaluate and another that explains how to evaluate . The first taxonomy identifies key components of LLM-based agents for multi-turn conversations and their evaluation dimensions, including task completion, response quality, user experience, memory and context retention, as well as planning and tool integration. These components ensure that the performance of conversational agents is assessed in a holistic and meaningful manner. The second taxonomy system focuses on the evaluation methodologies. It categorizes approaches into annotation-based evaluations, automated metrics, hybrid strategies that combine human assessments with quantitative measures, and self-judging methods utilizing LLMs. This framework not only captures traditional metrics derived from language understanding, such as BLEU and ROUGE scores, but also incorporates advanced techniques that reflect the dynamic, interactive nature of multi-turn dialogues. Together, these frameworks summarize the current status quo, expose limitations in traditional practices, and provide a structured blueprint for improvement. Based on the summarization of existing studies, we identify several challenges and propose future directions, including the development of scalable, real-time evaluation pipelines, enhanced privacy-preserving mechanisms, and robust metrics that capture dynamic multi-turn interactions. Our contributions bridge historical insights with modern practices, paving the way for next-generation, reliably evaluated conversational AI systems and offering a comprehensive guide for researchers and practitioners.

Cloud-based AI systems offer significant benefits but also introduce vulnerabilities, making deep neural network (DNN) models susceptible to malicious tampering. This tampering may involve harmful behavior injection or resource reduction, compromising model integrity and performance. To detect model tampering, hard-label fingerprinting techniques generate sensitive samples to probe and reveal tampering. Existing fingerprinting methods are mainly based on gradient-defined sensitivity or decision boundary, with the latter showing a manifest superior detection performance. However, all existing fingerprinting methods either suffer from insufficient sensitivity or incur high computational costs.In this paper, we theoretically analyze the black-box co-optimal tampering detection sensitivity of fingerprint samples in the context of decision boundary and gradient-defined sensitivity. Based on this, we further propose Steep-Decision-Boundary Fingerprinting (SDBF), a novel lightweight approach for hard-label tampering detection that inherently and efficiently combines the strengths of existing fingerprinting techniques. SDBF places fingerprint samples near the steep decision boundary, where the outputs of samples are inherently highly sensitive to tampering. We also design a Max Boundary Coverage Strategy (MBCS), which enhances samples’ diversity over the decision boundary. Theoretical analysis and extensive experimental results show that SDBF outperforms existing SOTA hard-label fingerprinting methods in both sensitivity and efficiency.

Adversarial attacks threaten the integrity of deep neural networks (DNNs), particularly in high-stakes applications. In this paper, we present a novel black-box adversarial attack that leverages the diverse checkpoints generated during a single model’s training trajectory. Unlike conventional ensemble attacks that require multiple surrogate models with diverse architectures, our approach exploits the intrinsic diversity captured over different training stages of a single surrogate model. By decomposing the learned representations into task-intrinsic and task-irrelevant components, we employ an accuracy gap-based selection strategy to identify checkpoints that predominantly capture transferable, task-intrinsic knowledge. Extensive experiments on ImageNet and CIFAR-10 demonstrate that our method consistently outperforms traditional ensemble attacks in terms of transferability, even under resource-constrained and practical settings. This work offers a resource-efficient solution for crafting highly transferable adversarial examples and provides new insights into the dynamics of adversarial vulnerability.

The proliferation of Machine Learning as a Service (MLaaS) has enabled widespread deployment of large language models (LLMs) via cloud APIs, but also raises critical concerns about model integrity and security.Existing black-box tamper detection methods, such as watermarking and fingerprinting, rely on the stability of model outputs-a property that does not hold for inherently stochastic LLMs.We address this challenge by formulating blackbox tamper detection for LLMs as a hypothesistesting problem.To enable efficient and sensitive fingerprinting, we derive a first-order surrogate for KL divergence-the entropy-gradient norm-to identify prompts most responsive to parameter perturbations.Building on this, we propose Regularized Entropy-Sensitive Fingerprinting (RESF), which enhances sensitivity while regularizing entropy to improve output stability and control false positives.To further distinguish tampering from benign randomness, such as temperature shifts, RESF employs a lightweight two-tier sequential test combining support-based and distributional checks with rigorous false-alarm control.Comprehensive analysis and experiments across multiple LLMs show that RESF achieves up to 98.80% detection accuracy under challenging conditions, such as minimal LoRA fine-tuning with five optimized fingerprints.RESF consistently demonstrates strong sensitivity and robustness, providing an effective and scalable solution for black-box tamper detection in cloud-deployed LLMs.

Reinforcement Learning (RL) has emerged as a powerful tool for neural combinatorial optimization, enabling models to learn heuristics that solve complex problems without requiring expert knowledge. Despite significant progress, existing RL approaches face challenges such as diminishing reward signals and inefficient exploration in vast combinatorial action spaces, leading to inefficiency. In this paper, we propose **Preference Optimization**, a novel method that transforms quantitative reward signals into qualitative preference signals via statistical comparison modeling, emphasizing the superiority among sampled solutions. Methodologically, by reparameterizing the reward function in terms of policy and utilizing preference models, we formulate an entropy-regularized RL objective that aligns the policy directly with preferences while avoiding intractable computations. Furthermore, we integrate local search techniques into the fine-tuning rather than post-process to generate high-quality preference pairs, helping the policy escape local optima. Empirical results on various benchmarks, such as the Traveling Salesman Problem (TSP), the Capacitated Vehicle Routing Problem (CVRP) and the Flexible Flow Shop Problem (FFSP), demonstrate that our method significantly outperforms existing RL algorithms, achieving superior convergence efficiency and solution quality.

The integration of large language models (LLMs) with external content has enabled applications such as Microsoft Copilot but also introduced vulnerabilities to indirect prompt injection attacks. In these attacks, malicious instructions embedded within external content can manipulate LLM outputs, causing deviations from user expectations. To address this critical yet under-explored issue, we introduce the first benchmark for bindirect prompt injection attacks, named BIPIA, to assess the risk of such vulnerabilities. Using BIPIA, we evaluate existing LLMs and find them universally vulnerable. Our analysis identifies two key factors contributing to their success: LLMs' inability to distinguish between informational context and actionable instructions, and their lack of awareness in avoiding the execution of instructions within external content. Based on these findings, we propose two novel defense mechanisms -- boundary awareness and explicit reminder -- to address these vulnerabilities in both black-box and white-box settings. Extensive experiments demonstrate that our black-box defense provides substantial mitigation, while our white-box defense reduces the attack success rate to near-zero levels, all while preserving the output quality of LLMs. We hope this work inspires further research into securing LLM applications and fostering their safe and reliable use. Our code is available at https://github.com/microsoft/BIPIA.

Adversarial examples crafted on one model often exhibit poor transferability to others, hindering their effectiveness in black-box settings. This limitation arises from two key factors: (i) \emph{decision-boundary variation} across models and (ii) \emph{representation drift} in feature space. We address these challenges through a new perspective that frames transferability for \emph{untargeted attacks} as a \emph{consensus-robust optimization} problem: adversarial perturbations should remain effective across a neighborhood of plausible target models. To model this uncertainty, we introduce two complementary perturbation channels: a \emph{parameter channel}, capturing boundary shifts via weight perturbations, and a \emph{representation channel}, addressing feature drift via stochastic blending of clean and adversarial activations. We then propose \emph{CORTA} (COnsensus--Robust Transfer Attack), a lightweight attack instantiated from this robust formulation using two first-order strategies: (i) sensitivity regularization based on the squared Frobenius norm of logits’ Jacobian with respect to weights, and (ii) Monte Carlo sampling for blended feature representations. Our theoretical analysis provides a certified lower bound linking these approximations to the robust objective. Extensive experiments on CIFAR-100 and ImageNet show that CORTA significantly outperforms state-of-the-art transfer-based methods---including ensemble approaches---across CNN and Vision Transformer targets. Notably, CORTA achieves a \emph{19.1 percentage-point gain in transfer success rate over the best prior method} while using only a single surrogate model.

Sheng Yang, Yurong Wu, Yan Gao, Zineng Zhou, Bin Benjamin Zhu, Xiaodi Sun, Jian-Guang Lou, Zhiming Ding, Anbang Hu, Yuan Fang, Yunsong Li, Junyan Chen, Linjun Yang. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

In our increasingly interconnected digital world, face recognition serves as a vital security layer for identity verification. However, impersonation attacks pose a significant threat to face recognition systems. While adversarial attacks have proven effective in impersonation, current methods primarily target face recognition, overlooking other crucial processes in real-world applications, such as action-based liveness detection.To address this gap, we introduce MysticMask, a novel adversarial mask attack designed to penetrate the entire face recognition pipeline for impersonation. MysticMask operates in the 3D domain, leveraging foldable medical face masks that automatically align with facial landmarks, enabling accurate physical simulations. MysticMask attacks all processes in a face recognition system simultaneously, including face detection, action-based liveness detection, and face recognition. Additionally, a landmark alignment technique is proposed to pass action-based liveness detection. Extensive experiments conducted in both simulated 3D and real-world scenarios demonstrate MysticMask’s superiority over state-of-the-art methods.

Cloud-based AI services offer numerous benefits but also introduce vulnerabilities, allowing for tampering with deployed DNN models, ranging from injecting malicious behaviors to reducing computing resources. Fingerprint samples are generated to query models to detect such tampering. In this paper, we present Intersecting-Boundary-Sensitive Fingerprinting (IBSF), a novel method for black-box integrity verification of DNN models using only top-1 labels. Recognizing that tampering with a model alters its decision boundary, IBSF crafts fingerprint samples from normal samples by maximizing the partial Shannon entropy of a selected subset of categories to position the fingerprint samples near decision boundaries where the categories in the subset intersect. These fingerprint samples are almost indistinguishable from their source samples. We theoretically establish and confirm experimentally that these fingerprint samples’ expected sensitivity to tampering increases with the cardinality of the subset. Extensive evaluation demonstrates that IBSF surpasses existing state-of-the-art fingerprinting methods, particularly with larger subset cardinality, establishing its state-of-the-art performance in black-box tampering detection using only top-1 labels. The IBSF code is available at https://github.com/CGCL-codes/IBSF.

Generative adversarial networks (GANs) have remarkably advanced in diverse domains, especially image generation and editing. However, the misuse of GANs for generating deceptive images, such as face replacement, raises significant security concerns, which have gained widespread attention. Therefore, it is urgent to develop effective detection methods to distinguish between real and fake images. Current research centers around the application of transfer learning. Nevertheless, it encounters challenges such as knowledge forgetting from the original dataset and inadequate performance when dealing with imbalanced data during training. To alleviate this issue, this paper introduces a novel GAN-generated image detection algorithm called X-Transfer, which enhances transfer learning by utilizing two neural networks that employ interleaved parallel gradient transmission. In addition, we combine AUC loss and cross-entropy loss to improve the model’s performance. We carry out comprehensive experiments on multiple facial image datasets. The results show that our model outperforms the general transferring approach, and the best metric achieves 99.04%, which is increased by approximately 10%. Furthermore, we demonstrate excellent performance on non-face datasets, validating its generality and broader application prospects.

Cloud-based machine learning services offer significant advantages but also introduce the risk of tampering with cloud-deployed deep neural network (DNN) models. Black-box integrity verification (BIV) allows model owners and end-users to determine if a cloud-deployed DNN model has been tampered with by examining only the top-1 label responses. Fingerprinting generates fingerprint samples to query the model, achieving BIV with no impact on the model's accuracy. In this paper, we present BIVBench, the first comprehensive benchmark for BIV of DNN models. BIVBench covers 16 types of model modifications, providing extensive coverage of practical modification scenarios. Our analysis reveals that existing fingerprinting methods, which are typically focused on significant tampering, lack the sensitivity needed to effectively detect subtle yet common and potentially severe modifications. To address this limitation, we propose MiSentry (Model Integrity Sentry), a novel fingerprinting method that leverages meta-learning. MiSentry strategically incorporates a few subtly modified models into the meta-learning model zoo and maximizes the divergence of output predictions between the target model and the modified models in the model zoo to generate highly sensitive, generalizable, and effective fingerprint samples. Extensive evaluations using BIVBench demonstrate that MiSentry outperforms existing state-of-the-art methods overall and significantly surpasses them in detecting subtle modifications. The BIVBench and supplementary materials are available at: https://github.com/CGCL-codes/BIVBench.

RGB-Thermal Salient Object Detection (RGBT-SOD) plays a critical role in complex scene recognition applications, such as autonomous driving. However, security research in this domain is still in its infancy. This paper presents the first backdoor attack on RGBT-SOD systems, generating saliency maps on triggered inputs that depict non-existent salient objects chosen by the attacker or falsely mark an entire image as fully salient or entirely non-salient. We uncover that triggers have an influence range for generating non-existent salient objects, supported by a theoretical analysis. Extensive experiments show the effectiveness of our attack in both digital and physical-world scenarios. Notably, our dual-modality backdoor attack achieves an Attack Success Rate (ASR) of 86.72% with only five pairs of poisoned images in model training. After investigating potential countermeasures, we find them inadequate in mitigating our attacks, highlighting the urgent need for robust defenses against sophisticated backdoor attacks in RGBT-SOD systems.

Wenjun Peng, Jingwei Yi, Fangzhao Wu, Shangxi Wu, Bin Bin Zhu, Lingjuan Lyu, Binxing Jiao, Tong Xu, Guangzhong Sun, Xing Xie. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Federated learning is used to train a shared model in a decentralized way without clients sharing private data with each other. Federated learning systems are susceptible to poisoning attacks when malicious clients send false updates to the central server. Existing defense strategies are ineffective under non-IID data settings. This paper proposes a new defense strategy, FedCPA (Federated learning with Critical Parameter Analysis). Our attack-tolerant aggregation method is based on the observation that benign local models have similar sets of top-k and bottom-k critical parameters, whereas poisoned local models do not. Experiments with different attack scenarios on multiple datasets demonstrate that our model outperforms existing defense strategies in defending against poisoning attacks.

Controlling the degree of stylization in the Neural Style Transfer (NST) is a little tricky since it usually needs hand-engineering on hyper-parameters. In this paper, we propose the first deep Reinforcement Learning (RL) based architecture that splits one-step style transfer into a step-wise process for the NST task. Our RL-based method tends to preserve more details and structures of the content image in early steps, and synthesize more style patterns in later steps. It is a user-easily-controlled style-transfer method. Additionally, as our RL-based model performs the stylization progressively, it is lightweight and has lower computational complexity than existing one-step Deep Learning (DL) based models. Experimental results demonstrate the effectiveness and robustness of our method.

As a popular distributed privacy-preserving machine learning approach, federated learning (FL) trains a global model across numerous clients without directly sharing their private training data. A fundamental challenge of FL is non-independently and identically distributed (non-IID) data, which can be addressed by clustered FL (CFL) that explores inherent partitions among clients to capture heterogeneous local data distributions. Existing CFL methods, however, tend to optimise the mean accuracy across all clients, overlooking the important issue of fairness, that is, uniformity of clients' performance. Besides, prior FL- related visual analytics (VA) frameworks are developed based on the standard FL scenario of single global model and are hence inapplicable to the CFL case of multiple global (cluster) models. To fill the research gap, this study proposes a novel VA framework named cCFLvis to illustrate and analyse variation of fairness during the CFL learning process. A new metric, change of absolute deviation from the mean (ΔAD), is introduced to quantify such variation. Client embeddings are generated by contrastive learning-based dimensionality reduction techniques to facilitate 2-d visualisation of client disparity. Visual cues can be drawn to help understand fairness variation and to offer clues for improving learning outcomes. cCFL vis is a general framework applicable to various CFL methods. Its effectiveness is showcased in a simple demonstration and two case studies, each conducted with a distinct representative CFL method and a different federated non-IID dataset.

Reinforcement learning is used to tackle complex tasks with high-dimensional sensory inputs. Over the past decade, a wide range of reinforcement learning algorithms have been developed, with recent progress benefiting from deep learning for raw sensory signal representation. This raises a natural question: how well do these algorithms perform across different robotic manipulation tasks? To objectively compare algorithms, benchmarks use performance metrics. Benchmarks use objective performance metrics to offer a scientific way to compare algorithms. In this paper, we introduce RMBench, the first benchmark for robotic manipulations with high-dimensional continuous action and state spaces. We implement and evaluate reinforcement learning algorithms that take observed pixels as inputs and report their average performance and learning curves to demonstrate their performance and training stability. Our study concludes that none of the evaluated algorithms can handle all tasks well, with soft Actor-Critic outperforming most algorithms in terms of average reward and stability, and an algorithm combined with data augmentation potentially facilitating learning policies. Our code is publicly available at https://github.com/xiangyanfei212/RMBench-2022.git, including all benchmark tasks and studied algorithms.

Federated learning enables learning from decentralized data sources without compromising privacy, which makes it a crucial technique. However, it is vulnerable to model poisoning attacks, where malicious clients interfere with the training process. Previous defense mechanisms have focused on the server-side by using careful model aggregation, but this may not be effective when the data is not identically distributed or when attackers can access the information of benign clients. In this paper, we propose a new defense mechanism that focuses on the client-side, called FedDefender, to help benign clients train robust local models and avoid the adverse impact of malicious model updates from attackers, even when a server-side defense cannot identify or remove adversaries. Our method consists of two main components: (1) attack-tolerant local meta update and (2) attack-tolerant global knowledge distillation. These components are used to find noise-resilient model parameters while accurately extracting knowledge from a potentially corrupted global model. Our client-side defense strategy has a flexible structure and can work in conjunction with any existing server-side strategies. Evaluations of real-world scenarios across multiple datasets show that the proposed method enhances the robustness of federated learning against model poisoning attacks.

News recommendation is essential for personalized news distribution. Federated news recommendation, which enables collaborative model learning from multiple clients without sharing their raw data, is a promising approach for preserving users' privacy. However, the security of federated news recommendation is still unclear. In this paper, we study this problem by proposing an untargeted attack on federated news recommendation called UA-FedRec. By exploiting the prior knowledge of news recommendation and federated learning, UA-FedRec can effectively degrade the model performance with a small percentage of malicious clients. First, the effectiveness of news recommendation highly depends on user modeling and news modeling. We design a news similarity perturbation method to make representations of similar news farther and those of dissimilar news closer to interrupt news modeling, and propose a user model perturbation method to make malicious user updates in opposite directions of benign updates to interrupt user modeling. Second, updates from different clients are typically aggregated with a weighted average based on their sample sizes. We propose a quantity perturbation method to enlarge sample sizes of malicious clients in a reasonable range to amplify the impact of malicious updates. Extensive experiments on two real-world datasets show that UA-FedRec can effectively degrade the accuracy of existing federated news recommendation methods, even when defense is applied. Our study reveals a critical security issue in existing federated news recommendation systems and calls for research efforts to address the issue. Our code is available at https://github.com/yjw1029/UA-FedRec.

Labeled datasets are essential for supervised machine learning. Various data labeling tools have been built to collect labels in different usage scenarios. However, developing labeling tools is time-consuming, costly, and expertise-demanding on software development. In this paper, we propose a conceptual framework for data labeling and OneLabeler based on the conceptual framework to support easy building of labeling tools for diverse usage scenarios. The framework consists of common modules and states in labeling tools summarized through coding of existing tools. OneLabeler supports configuration and composition of common software modules through visual programming to build data labeling tools. A module can be a human, machine, or mixed computation procedure in data labeling. We demonstrate the expressiveness and utility of the system through ten example labeling tools built with OneLabeler. A user study with developers provides evidence that OneLabeler supports efficient building of diverse data labeling tools.

In this paper, we propose a new few-shot object detection (FSOD) framework that introduces a new contrastive branch to extract the class representation of images, which improves the generalization performance of the detection model for novel classes. Additionally, we investigate the effectiveness of both self-supervised and supervised contrastive losses for class-specific encoding in our framework. Experimental results on the benchmark datasets indicate that our proposed method archives the state-of-the-art performance compared with existing FSOD methods.

Deep learning-based image embedding methods are typically designed for natural images and may not work for chart images due to their homogeneous regions, which lack variations to hide data both robustly and imperceptibly. In this paper, we propose ChartStamp, the first chart embedding method that is robust to real-world printing and displaying (printed on paper and displayed on screen, respectively, and then captured with a camera) while maintaining a good perceptual quality. ChartStamp hides 100, 1,000, or 10,000 raw bits into a chart image, depending on the designated robustness to printing, displaying, or JPEG. To ensure perceptual quality, it introduces a new perceptual model to guide embedding to insensitive regions of a chart image and a smoothness loss to ensure smoothness of the embedding residual in homogeneous regions. ChartStamp applies a distortion layer approximating designated real-world manipulations to train a model robust to these manipulations. Our experimental evaluation indicates that ChartStamp achieves the robustness and embedding capacity on chart images similar to their state-of-the-art counterparts on natural images. Our user studies indicate that ChartStamp achieves better perceptual quality than existing robust chart embedding methods and that our perceptual model outperforms the existing perceptual model.

In the cloud era, a large amount of data is uploaded to and processed by public clouds. The risk of privacy leakage has become a major concern for cloud users. Cloud-based video surveillance requires motion detection, which may reveal the privacy of people in a surveillance video. Privacy-preserving video surveillance allows motion detection while protecting privacy. The existing scheme [ 25 ], designed to detect motion on encrypted and H.264-compressed surveillance videos, does not work well on more advanced video compression schemes such as HEVC. In this article, we propose the first motion detection method on encrypted and HEVC-compressed videos. It adopts a novel approach that exploits inter-prediction reference relationships among coding blocks to detect motion regions. The partition pattern and the number of coding bits of each detection block used in prior art are also used to help detect motion regions. Spatial and temporal consistency of a moving object and Kalman filtering are applied to segment connected/merged motion regions, remove noise and background motions, and refine trajectories and shapes of detected moving objects. Experimental results indicate that our detection method achieves high detection recall, precision, and F1-score for surveillance videos of both high and low resolutions with various scenes. It has a similarly high detection accuracy on encrypted and HEVC-compressed videos as that of the existing motion detection method [ 25 ] on encrypted and H.264-compressed videos. Our proposed method incurs no bit-rate overhead and has a very low computational complexity for both motion detection and encryption of HEVC videos.

Deep neural networks (DNNs) are vulnerable to adversarial attacks. A great effort has been directed to developing effective defenses against adversarial attacks and finding vulnerabilities of proposed defenses. A recently proposed defense called Trapdoor-enabled Detection (TeD) deliberately injects trapdoors into DNN models to trap and detect adversarial examples targeting categories protected by TeD. TeD can effectively detect existing state-of-the-art adversarial attacks. In this paper, we propose a novel black-box adversarial attack on TeD, called Feature-Indistinguishable Attack (FIA). It circumvents TeD by crafting adversarial examples indistinguishable in the feature (i.e., neuron-activation) space from benign examples in the target category. To achieve this goal, FIA jointly minimizes the distance to the expectation of feature representations of benign samples in the target category and maximizes the distances to positive adversarial examples generated to query TeD in the preparation phase. A constraint is used to ensure that the feature vector of a generated adversarial example is within the distribution of feature vectors of benign examples in the target category. Our extensive empirical evaluation with different configurations and variants of TeD indicates that our proposed FIA can effectively circumvent TeD. FIA opens a door for developing much more powerful adversarial attacks. The FIA code is available at: https://github.com/CGCL-codes/FeatureIndistinguishableAttack.

Fooling people with highly realistic fake images generated with Deepfake or GANs brings a great social disturbance to our society. Many methods have been proposed to detect fake images, but they are vulnerable to adversarial perturbations – intentionally designed noises that can lead to the wrong prediction. Existing methods of attacking fake image detectors usually generate adversarial perturbations to perturb almost the entire image. This is redundant and increases the perceptibility of perturbations. In this paper, we propose a novel method to disrupt the fake image detection by determining key pixels to a fake image detector and attacking only the key pixels, which results in the L0and the L2norms of adversarial perturbations much less than those of existing works. Experiments on two public datasets with three fake image detectors indicate that our proposed method achieves state-of the-art performance in both white-box and black-box attacks.

Deep neural networks have been demonstrated to be vulnerable to adversarial attacks: subtle perturbation can completely change prediction result. The vulnerability has led to a surge of research in this direction, including adversarial attacks on object detection networks. However, previous studies are dedicated to attacking anchor-based object detectors. In this paper, we present the first adversarial attack on anchor-free object detectors. It conducts category-wise, instead of previously instance-wise, attacks on object detectors, and leverages high-level semantic information to efficiently generate transferable adversarial examples, which can also be transferred to attack other object detectors, even anchor-based detectors such as Faster R-CNN. Experimental results on two benchmark datasets demonstrate that our proposed method achieves state-of-the-art performance and transferability.

In practice, charts are widely stored as bitmap images. Although easily consumed by humans, they are not convenient for other uses. For example, changing the chart style or type or a data value in a chart image practically requires creating a completely new chart, which is often a time-consuming and error-prone process. To assist these tasks, many approaches have been proposed to automatically extract information from chart images with computer vision and machine learning techniques. Although they have achieved promising preliminary results, there are still a lot of challenges to overcome in terms of robustness and accuracy. In this paper, we propose a novel alternative approach called Chartem to address this issue directly from the root. Specifically, we design a data-embedding schema to encode a significant amount of information into the background of a chart image without interfering human perception of the chart. The embedded information, when extracted from the image, can enable a variety of visualization applications to reuse or repurpose chart images. To evaluate the effectiveness of Chartem, we conduct a user study and performance experiments on Chartem embedding and extraction algorithms. We further present several prototype applications to demonstrate the utility of Chartem.

Homomorphic Encryption (HE) allows processing cipher-text data, but it is a challenge to enable complex methods such as multimedia decompression in the HE domain. In this paper, we propose a novel scheme to enable FLAC (Free Lossless Audio Codec) decompression in the HE domain. FLAC applies linear prediction to predict the current sample and Golomb coding to encode residuals. FLAC decoding relies heavily on dynamic controls that HE does not support due to unknown values of control variables after encryption. Our scheme regularizes dynamic controls in FLAC decoding with static controls by calculating an encrypted matching bit for each possible value of a control variable and producing candidate results as if it were a match. The summation of each possible value’s candidate results multiplied by its matching bit is equivalent to selecting the results of the matched control value. Our FLAC decoding scheme enables Single-Instruction Multiple-Data (SIMD): multiple (e.g., 256) plaintexts are packed and encrypted into a single ciphertext, and decoding one encrypted frame corresponds to decoding multiple plaintext frames. Our scheme is applicable to other audio compression standards based on similar technologies. Experimental results are also reported.

With a growing demand of concurrent software to exploit multi-core hardware capability, concurrency vulnerabilities have become an inevitable threat to the security of today's IT industry. Existing concurrent program detection schemes focus mainly on detecting concurrency errors such as data races, atomicity violation, etc., with little attention paid to detect concurrency vulnerabilities that may be exploited to infringe security. In this paper, we propose a heuristic framework that combines both static analysis and fuzz testing to detect targeted concurrency vulnerabilities such as concurrency buffer overflow, double free, and use-after-free. The static analysis locates sensitive concurrent operations in a concurrent program, categorizes each finding into a potential type of concurrency vulnerability, and determines the execution order of the sensitive operations in each finding that would trigger the suspected concurrency vulnerability. The results are then plugged into the fuzzer with the execution order fixed by the static analysis in order to trigger the suspected concurrency vulnerabilities.

Cloud-based video surveillance requires protecting privacy yet allowing cloud to perform motion detection and tracking. Prior art allows performing surveillance on encrypted videos but details of trajectories of moving objects are exposed. In this paper, we propose a video surveillance system that supports adjustable granularity of motion detection and tracking for fine-grained control on conflicting requirements between privacy protection and accuracy of motion detection and tracking. Our encryption adds a permutation layer to conventional format-compliant selective encryption to ensure motion information can be recovered only at a given granularity yet incurs a very small impact on the bitrate and processing speed of video compression. Our motion detection method estimates both local and global motions to distinguish foreground motions from background motions and deduce trajectories of foreground moving objects. Experimental results indicate that our system has fulfilled the design goal.

Privacy-preserving processing is desirable for cloud computing to relieve users' concern of loss of control of their uploaded data. This may be fulfilled with homomorphic encryption. With widely used JPEG, it is desirable to enable JPEG decompression in the homomorphic encryption domain. This is a great challenge since JPEG decoding needs to determine a matched codeword, which then extracts a codeword-dependent number of coefficients. With no access to the information of encrypted content, a decoder does not know which codeword is matched, and thus cannot tell how many coefficients to extract, not to mention to compute their values. In this paper, we propose a novel scheme that enables JPEG decompression in the homomorphic encryption domain. The scheme applies a statically controlled iterative procedure to decode one coefficient per iteration. In one iteration, each codeword is compared with the bitstream to compute an encrypted Boolean that represents if the codeword is a match or not. Each codeword would produce an output coefficient and generate a new bitstream by dropping consumed bits as if it were a match. If a codeword is associated with more than one coefficient, the codeword is replaced with the codeword representing the remaining undecoded coefficients for the next decoding iteration. The summation of each codeword's output multiplied by its matching Boolean is the output of the current iteration. This is equivalent to selecting the output of a matched codeword. A side benefit of our statically controlled decoding procedure is that paralleled Single-Instruction Multiple-Data (SIMD) is fully supported, wherein multiple plaintexts are encrypted into a single plaintext, and decoding a ciphertext block corresponds to decoding all corresponding plaintext blocks. SIMD also reduces the total size of ciphertexts of an image. Experimental results are reported to show the performance of our proposed scheme.

We propose a novel concept and a model of image point memorability (IPM) for analyzing click-based graphical passwords that have been studied extensively in both the security and HCI communities. In our model, each point in an image is associated with a numeric index that indicates the point's memorability level. This index can be approximated either by automatic computer vision algorithms or via human assistance. Using our model, we can rank-order image points by their relative memorability with a decent accuracy. We show that the IPM model has both defensive and offensive applications. On the one hand, we apply the model to generate high-quality graphical honeywords. This is the first work on honeywords for graphical passwords, whereas all previous methods are only for generating text honeywords and thus inapplicable. On the other hand, we use the IPM model to develop the first successful dictionary attacks on Persuasive Cued Click Points (PCCP), which is the state-of-the-art click-based graphical password scheme and robust to all prior dictionary attacks. We show that the probability distribution of PCCP passwords is seriously biased when it is examined with the lens of the IPM model. Although PCCP was designed to generate random passwords, its effective password space as we measured can be as small as 30.58 bits, which is substantially weaker than its theoretical and commonly believed strength (43 bits). The IPM model is applicable to all click-based graphical password schemes, and our analyses can be extended to other graphical passwords as well.

Many security primitives are based on hard mathematical problems. Using hard AI problems for security is emerging as an exciting new paradigm, but has been under-explored. In this paper, we present a new security primitive based on hard AI problems, namely, a novel family of graphical password systems built on top of Captcha technology, which we call Captcha as graphical passwords (CaRP). CaRP is both a Captcha and a graphical password scheme. CaRP addresses a number of security problems altogether, such as online guessing attacks, relay attacks, and, if combined with dual-view technologies, shoulder-surfing attacks. Notably, a CaRP password can be found only probabilistically by automatic online guessing attacks even if the password is in the search set. CaRP also offers a novel approach to address the well-known image hotspot problem in popular graphical password systems, such as PassPoints, that often leads to weak password choices. CaRP is not a panacea, but it offers reasonable security and usability and appears to fit well with some practical applications for improving online security.

Phishing is the third cyber-security threat globally and the first cyber-security threat in China. There were 61.69 million phishing victims in China alone from June 2011 to June 2012, with the total annual monetary loss more than 4.64 billion US dollars. These phishing attacks were highly concentrated in targeting at a few major Websites. Many phishing Webpages had a very short life span. In this paper, we assume the Websites to protect against phishing attacks are known, and study the effectiveness of machine learning based phishing detection using only lexical and domain features, which are available even when the phishing Webpages are inaccessible. We propose several novel highly effective features, and use the real phishing attack data against Taobao and Tencent, two main phishing targets in China, in studying the effectiveness of each feature, and each group of features. We then select an optimal set of features in our phishing detector, which has achieved a detection rate better than 98%, with a false positive rate of 0.64% or less. The detector is still effective when the distribution of phishing URLs changes.

Discretization is a standard technique used in click-based graphical passwords for tolerating input variance so that approximately correct passwords are accepted by the system. In this paper, we show for the first time that two representative discretization schemes leak a significant amount of password information, undermining the security of such graphical passwords. We exploit such information leakage for successful dictionary attacks on Persuasive Cued Click Points (PCCP), which is to date the most secure click-based graphical password scheme and was considered to be resistant to such attacks. In our experiments, our purely automated attack successfully guessed 69.2% of the passwords when Centered Discretization was used to implement PCCP, and 39.4% of the passwords when Robust Discretization was used. Each attack dictionary we used was of approximately 235 entries, whereas the full password space was of 243 entries. For Centered Discretization, our attack still successfully guessed 50% of the passwords when the dictionary size was reduced to approximately 230 entries. Our attack is also applicable to common implementations of other click-based graphical password systems such as PassPoints and Cued Click Points -- both have been extensively studied in the research communities.

We systematically study the design of image recognition CAPTCHAs (IRCs) in this paper. We first review and examine all existing IRCs schemes and evaluate each scheme against the practical requirements in CAPTCHA applications, particularly in large-scale real-life applications such as Gmail and Hotmail. Then we present a security analysis of the representative schemes we have identified. For the schemes that remain unbroken, we present our novel attacks. For the schemes for which known attacks are available, we propose a theoretical explanation why those schemes have failed. Next, we provide a simple but novel framework for guiding the design of robust IRCs. Then we propose an innovative IRC called Cortcha that is scalable to meet the requirements of large-scale applications. It relies on recognizing objects by exploiting the surrounding context, a task that humans can perform well but computers cannot. An infinite number of types of objects can be used to generate challenges, which can effectively disable the learning process in machine learning attacks. Cortcha does not require the images in its image database to be labeled. Image collection and CAPTCHA generation can be fully automated. Our usability studies indicate that, compared with Google's text CAPTCHA, Cortcha allows a slightly higher human accuracy rate but on average takes more time to solve a challenge.

This paper presents smart caching schemes for Web browsers. For modern Web applications, the style formatting and layout calculation often account for substantial amounts of the local computation in order to render a Web page. In this paper, we propose two caching schemes to reduce the computation of style formatting and layout calculation, named smart style caching and layout caching, respectively. The stable style data and layout data for DOM (Document Object Model) elements are recorded to construct the caches when a Web page is browsed. The cached data is checked in the granularity of DOM elements and applied directly if the identified DOM element is not changed in the sequent visits to the same page.

In this paper, a benchmark called WPBench is reported to evaluate the responsiveness of Web browsers for modern Web 2.0 applications. In WPBench, variations of servers and networks are removed and the benchmark result is the closest to what Web users would perceive. To achieve these, WPBench records users' interactions with typical Web 2.0 applications, and then replays Web navigations when benchmarking browsers. The replay mechanism can emulate the actual user interactions and the characteristics of the servers and the networks in a consistent way independent of browsers so that any browser compliant to the standards can be benchmarked fairly. In addition to describing the design and generation of WPBench, we also report the WPBench comparison results on the responsiveness performance for three popular Web browsers: Internet Explorer, Firefox and Chrome.

Digital rights management (DRM) is widely used to protect intellectual property for content owners but consumer privacy is sacrificed. A user's playing statistics can be collected by the client DRM module and the license server. In this paper, we propose a DRM system in which the license server can generate the content decryption key for a user to play an encrypted content object without gaining any information to link to the specific content object encrypted by the content encryption key. This is achieved by applying a (partially) blind signature primitive in the license acquisition protocol and by adopting a key scheme that a content encryption key depends on the information retrieved from the content object and a secret that only the license server knows. By requesting that the client DRM module does not send any information about a user's playing statistics and all the messages the client DRM module sends out are in plain text for easy checking by a user if the client DRM module abides by this rule, consumer privacy is fully protected in our DRM system.

In this position paper, we report the progress of our project to analyze security of the traitor tracing technology used in Advanced Access Content System (AACS). For a simplified problem that all sequence keys are statically assigned according to known Reed Solomon (RS) codes, and a colluder is to be identified with the traitor tracing, we present a mix-and-match colluding attack to victimize an innocent device such that using the highest score of matches with AACS' traitor tracing to identify a colluder will always indentify the victim as a colluder no matter how many movies are tested. Both theoretical analysis and experimental results show that a group of arbitrary 20 or more colluding devices out of a billion devices supported by AACS can always successfully victimize an innocent device. With 26 arbitrary colluders, an arbitrarily given device has a probability of 39.13% to be successfully victimized. Moreover, our attack enables everyone in a group as small as 6 arbitrary colluding devices to escape from being identified as a colluder with the traitor identification method. Analysis with more realistic assumptions will be reported in the future.

Image and video labeling is important for computers to understand images and videos and for image and video search. Manual labeling is tedious and costly. Automatically image and video labeling is yet a dream. In this paper, we adopt a Web 2.0 approach to labeling images and videos efficiently: Internet users around the world are mobilized to apply their "common sense" to solve problems that are hard for today's computers, such as labeling images and videos. We first propose a general human computation framework that binds problem providers, Web sites, and Internet users together to solve large-scale common sense problems efficiently and economically. The framework addresses the technical challenges such as preventing a malicious party from attacking others, removing answers from bots, and distilling human answers to produce high-quality solutions to the problems. The framework is then applied to labeling images. Three incremental refinement stages are applied. The first stage collects candidate labels of objects in an image. The second stage refines the candidate labels using multiple choices. Synonymic labels are also correlated in this stage. To prevent bots and lazy humans from selecting all the choices, trap labels are generated automatically and intermixed with the candidate labels. Semantic distance is used to ensure that the selected trap labels would be different enough from the candidate labels so that no human users would mistakenly select the trap labels. The last stage is to ask users to locate an object given a label from a segmented image. The experimental results are also reported in this paper. They indicate that our proposed schemes can successfully remove spurious answers from bots and distill human answers to produce high-quality image labels.

In this paper, client-site Web mashups are studied from component-oriented perspective, and CompoWeb, a component-oriented Web architecture, is proposed. In CompoWeb, a Web application is decomposed into Web components called gadgets. A gadget is an abstraction of functional or logical Web component. It is isolated from other gadgets for security and reliability. Contract-based channels are the only way to interact with each other. An abstraction of contract-based channels supported or required by a gadget is also presented. It enables binding of gadgets at deployment, and promotes interchangeable gadgets. Unlike the model of a normal function call where the function logic is executed in caller's context, CompoWeb ensures that the function logic is executed in callee's context so that both the caller and callee are protected. Implementation of a prototype CompoWeb system and its performance are also presented.

In many applications it is desired to save the content received from restricted broadcast or multicast to local files for subsequent replays. The locally saved files should be protected by a Digital Rights Management (DRM) system to prevent unauthorized usage. It is a great challenge to combine DRM with restricted broadcast or multicast since they are designed for different applications. In this paper, we first present two straight- forward solutions and discuss their drawbacks. We then present a novel content encryption key scheme for restricted broadcast and multicast that facilitates subsequent DRM protection for the saved content. It enables direct saving of encrypted content to local files and easy generation and management of decryption keys for replays of saved files. Only a single key needs to be delivered to a client in a license. Security of the proposed key scheme is analyzed, and comparison of the three methods in also discussed in this paper.

JPEG 2000 provides multiple scalable accesses to a single codestream. Digital Rights Management of a JPEG 2000 codestream should preserve the original flexibility of scalability yet provide a mechanism to ensure what you see is what you pay: a low resolution version displayed on a smart phone should pay less than a high resolution version displayed on a PC. We present an efficient key scheme for multi-type, multilevel scalable access control for JPEG 2000 and motion JPEG 2000 codestreams. The scheme is based on a poset representation of the scalable access control and a hash based hierarchical access key scheme, both proposed elsewhere. The proposed key scheme exploits the information contained in a codestream and the features invariant under truncations to minimize the file size overhead for DRM applications yet preserve correct derivation of keys for descendants even when an encrypted codestream is truncated.

An optimal source-channel packetization scheme for MPEG-4 fine granularity scalability (FGS) codestreams is proposed in this paper. The channel is modeled with a uniform error distribution to the enhancement layer transmission. A cost function that models the error expansion for a MPEG-4 FGS stream is derived, and then used in the optimal packetization problem subject to the same overhead as the conventional packetization scheme. An efficient scheme to find the optimal solution is described, which takes time similar to encoding an MPEG-4 FGS codestream. Experiments show that our scheme has up to 1.96 dB gain over the conventional packetization scheme.

An efficient syntax-compliant encryption scheme for JPEG 2000 and motion JPEG 2000 is proposed in this paper. Compressed visual data is completely encrypted yet the full scalability of the unencrypted codestream is completely preserved to allow near RD-optimal truncations and other manipulations securely without decryption. Compared with other reported schemes, our scheme shows advantages on syntax compliance, compression overhead, scalable granularity, and error resilience. In addition to preserving the original scalability, a JPEG 2000 codestream encrypted with our scheme has the same error resilience capability as the unencrypted codestream. The encrypted codestream is still syntax-compliant so that an encryption-unaware decoder can still decode the encrypted codestream, although the decoded visual data is completely garbled and meaningless. Our scheme has virtually no adverse impact on the compression efficiency.

The key management for multi-type, multi-level scalable access control of a fine granularity scalability codestream is addressed in this paper. We first present an efficient partially ordered set (poset) to represent scalable access control so that any access control schemes for a poset can be applied and a single secret key is needed to transfer to a user. We then present a secure key scheme modified from our previous scalable access key scheme for the resulting poset. The key scheme is based on the group Diffie-Hellman key agreement, and is secure

In this paper, we present an encryption scheme for MPEG-4 FGS which provides the same or a little coarser granularity of scalability after encryption. The scheme encrypts compressed data of each video packet or block independently. Initialization vectors are generated with a method to minimize the overhead. The scalability provided in an encrypted codestream using this scheme enables intermediate nodes to truncate an encrypted bitstream at near R-D optimality directly without decryption, which enhances system security. The scheme has virtually negligible overhead, and produces encrypted codestream with virtually the same error resilience performance as the unencrypted case. These features are very desirable in many applications

The newly adopted MPEG-4 fine granularity scalability (FGS) video coding standard offers easy and flexible adaptation to varying network bandwidths and different application needs. Encryption for FGS should preserve such adaptation capabilities and enable intermediate stages to process encrypted data directly without decryption. In this paper, we propose two novel encryption algorithms for MPEG-4 FGS that meet these requirements. The first algorithm encrypts an FGS stream (containing both the base and the enhancement layers) into a single access layer and preserves the original fine granularity scalability and error resilience performance in an encrypted stream. The second algorithm encrypts an FGS stream into multiple quality layers divided according to either peak signal-to-noise ratio (PSNR) or bit rates, with lower quality layers being accessible and reusable by a higher quality layer of the same type, but not vice versa. Both PSNR and bit-rate layers are supported simultaneously so a layer of either type can be selected on the fly without decryption. The base layer for the second algorithm may be unencrypted to allow free view of the content at low-quality or content-based search of a video database without decryption. Both algorithms are fast, error-resilient, and have negligible compression overhead. The same approach can be applied to other scalable multimedia formats.

The Yeung-Mintzer (Y-M) image authentication scheme has been well studied. Several vulnerabilities and modified schemes to fix them have been reported. We propose a novel oracle attack on the Y-M scheme and its variations. Our attack is very different from the previously proposed attacks. A single authenticated image plus access to a verifier (oracle) is enough in our attack. The verifier returns if a testing image is authentic or not. Locations of tampered pixels are not needed. To launch the attack, a single pixel is modified and the resulting image is sent to the verifier. Observation of outputs of the verifier is used to deduce the secret mapping functions and the embedded logo within an uncertainty of two possibilities. The deduced mapping functions are then used to modify the content of an authenticated image without detection or to authenticate an arbitrary image of the same size. Note that the logo is not used in the forgery so sophisticated protection of the logo cannot thwart the attack. Our attack is very efficient. Only 255 trials are needed to attack an 8-bit grayscale image and 765 trials for a 24-bit color image. The proposed attack can also be applied to attack pixel-wise variations of the Y-M scheme proposed to fix the previously reported vulnerabilities.

The recently proposed scalable multi-layer FGS (fine granularity scalability) encryption (SMLFE) encrypts an MPEG-4 FGS stream into multiple PSNR and bitrate quality layers for layered access control. Both layer types are supported simultaneously. A simple key scheme was used in SMLFE. We propose a novel key scheme for SMLFE that reduces the number of keys maintained and managed by a license server for each protected MPEG-4 FGS stream to two. The new key scheme needs only one key contained in a license to be sent to a consumer. This scheme is based on a cryptographic secure hash function and the Diffie-Hellman key agreement. It satisfies all the requirements of SMLFE and can be used to replace the original simple key scheme for SMLFE. The secure one-way hash and intractability of the Diffie-Hellman and the related problems of computing discrete logarithms ensure the security of the new key scheme.

MPEG-4 has recently adopted the fine granularity scalability (FGS) video coding technology which enables easy and flexible adaptation to bandwidth fluctuations and device capabilities. Encryption for FGS should preserve such adaptation capabilities and allow intermediate stages in the delivery to process the media on the ciphertext directly. In this paper, we propose a novel scalable access control scheme with this property for the MPEG-4 FGS format. It offers free browsing of the low-quality base layer video but controls the access to the enhancement layer at different service levels based on either PSNR or bitrates. Both types of service levels are supported simultaneously without jeopardizing each other's security. The scheme is fast and degrades neither compression efficiency nor error resilience of the MPEG-4 FGS. The approach is also applicable to other scalable multimedia.

Summary form only given. Data embedding algorithms embed binary streams in host multimedia signals. The embedded data can add features to the host multimedia signal or provide copyright protection. We review requirements for transparent data embedding techniques in audio signals. We describe and contrast current approaches to data embedding in audio. In particular, we emphasize the advantages and limitations of the various approaches. We also describe possible signal processing and protocol level attacks on audio watermarking algorithms. We conclude with a discussion of future research directions.

In this paper, we propose a novel adaptive arithmetic coding method that uses dual symbol sets: a primary symbol set that contains all the symbols that are likely to occur in the near future and a secondary symbol set that contains all other symbols. The simplest implementation of our method assumes that symbols that have appeared in the previously are highly likely to appear in the near future. It therefore fills the primary set with symbols that have occurred in the previously. Symbols move dynamically between the two symbol sets to adapt to the local statistics of the symbol source. The proposed method works well for sources, such as images, that are characterized by large alphabets and alphabet distributions that are skewed and highly nonstationary. We analyze the performance of the proposed method and compare it to other arithmetic coding methods, both theoretically and experimentally. We show experimentally that in certain contexts, e.g., with a wavelet-based image coding scheme that has appeared in the literature, the compression performance of the proposed method is better than that of the conventional arithmetic coding method and the zero-frequency escape arithmetic coding method.

We present a watermarking procedure to embed copyright protection into digital video. Our watermarking procedure is scene-based and video dependent. It directly exploits spatial masking, frequency masking, and temporal properties to embed an invisible and robust watermark. The watermark consists of static and dynamic temporal components that are generated from a temporal wavelet transform of the video scenes. The resulting wavelet coefficient frames are modified by a perceptually shaped pseudorandom sequence representing the author. The noise-like watermark is statistically undetectable to thwart unauthorized removal. Furthermore, the author representation resolves the deadlock problem. The multiresolution watermark may be detected on single frames without knowledge of the location of the frames in the video scene. We demonstrate the robustness of the watermarking procedure to several video degradations and distortions.

We introduce a watermarking procedure to embed copyright protection into digital video. Our video dependent watermarking procedure directly exploits the masking and temporal properties to embed an invisible and robust watermark. The watermark consists of static and dynamic temporal components that are generated from a temporal wavelet transform of the video scenes. To generate the watermark, the resulting wavelet coefficient frames are modified by a perceptually shaped pseudo-random sequence representing the author. The noise-like watermark is statistically undetectable to thwart unauthorized removal. Furthermore, the author representation resolves the deadlock problem. The multiresolution watermark may be detected on single frames without knowledge of the location of the frames in the video scene. We demonstrate the robustness of the watermarking procedure to several video distortions.

We introduce a scheme for hiding high bit-rate supplementary data, such as secondary video, into a digital video stream by directly modifying the pixels in the video frames. The technique requires no separate channel or bit interleaving to transmit the extra information. The data is invisibly embedded using a perception-based projection and quantization algorithm. The data hiding algorithm supports user-defined levels of accessibility and security. We illustrate our algorithm using examples of real-time video-in-video and speech-in-video. We also demonstrate the robustness of the data hiding procedure to video degradation and distortion, e.g., those that result from additive noise and compression.

We propose an image coding algorithm which employs data hiding techniques to "fold" an image into itself. Data hiding is a process of encoding extra information into a host image by making small modifications to its pixels. In our approach, an image is split into two parts of equal size: a host image and a residual image. The residual image is compressed into a bit stream and then embedded into the host image. The host image, which is 50% of the size of the original image, is coded using standard compression techniques. The embedded data does not increase the bit rate of the coded image. As a result, one may code only 50% pixels of the original image and still have perfect reconstruction. Experimental results indicate that the algorithm has a lot of potential in image coding.

We present a watermarking procedure to embed copyright protection into video sequences. To address issues associated with video motion and redundancy, individual watermarks are created for objects within the video. Each watermark is created by shaping an author and video dependent pseudo-random sequence according to the perceptual masking characteristics of the video. As a result, the watermark adapts to each video to ensure invisibility and robustness. Furthermore, the noise-like watermark is statistically undetectable. The watermark also resolves multiple ownership claims. We demonstrate the robustness of the watermarking procedure to video degradations, e.g., those that result from noise, MPEG compression, cropping, and printing and scanning.

We propose a watermarking scheme to hide copyright information in an image. The scheme employs visual masking to guarantee that the embedded watermark is invisible and to maximize the robustness of the hidden data. The watermark is constructed for arbitrary image blocks by filtering a pseudo-noise sequence (author id) with a filter that approximates the frequency masking characteristics of the visual system. The noise-like watermark is statistically invisible to deter unauthorized removal. Experimental results show that the watermark is robust to several distortions including white and colored noises, JPEG coding at different qualities, and cropping.

We propose a novel approach for low bit rate perceptually transparent image compression. It exploits both frequency and spatial visual masking effects and uses a combination of Fourier and wavelet transforms to encode different bands. Frequency domain masking is computed by using a fine to coarse analysis step. Spatial domain masking is computed either by using Girod's (1989) model or a coarse to fine analysis step that accurately computes local contrast. A discrete cosine transform is used in conjunction with frequency domain masking to encode the low frequency bands. The medium and high frequency bands are encoded using spatial domain masking and a wavelet transform. The encoding of these bands is based on a recursive selection of the important edges in each band. It uses cross-band prediction to minimize the bit rate. Experiments show the approach can achieve a very high quality to nearly transparent compression at bit rates of 0.2 to 0.4 bits/pixel.

The wavelet transform provides a multiresolution representation of images. Edges, which are visually important, produce large coefficients across several scales in the wavelet transform domain. By tracking and predicting these edge coefficients across scales in the wavelet transform domain, we can greatly improve the compressed image quality with little degradation in compression ratio. This paper proposes a novel model-based edge tracking and prediction in the wavelet domain. It separates textures from edges and codes them differently. Edges are coded via an edge tracking and prediction, while textures are coded with either ordinary wavelet based image coding techniques or a "wavelet-like" filter bank which is similar to the tuning channels in the human vision system. The coding noise is then coded with a noise modelling. Visual masking models are also used to ensure the compressed image has little or almost no perceptual distortion.