As a robust media representation technique, video hashing is frequently used in near-duplicate detection, video authentication, and antipiracy search. Distortions to a video may include spatial modifications to each frame, temporal de-synchronization, and joint spatio-temporal attacks. To address the increasingly difficult case of finding videos under spatio-temporal modifications, we propose a new framework called two-stage video hashing. First, an efficient automatic synchronization is achieved using dynamic time warping (DTW) and a complementary video comparison measure is developed based on flow hashing (FH), which is extracted from the synchronized videos. Next, a fusion mechanism called distance boosting is proposed to fuse the information extracted by DTW and FH in a future-proof manner in the sense whenever model retraining is needed, the existing hash vectors do not need to be regenerated. Experiments on real video collections show that such a hash extraction and fusion method enables unprecedented robustness under both spatial and temporal attacks.

Video hashing finds a wide array of applications in content authentication, robust retrieval and anti-piracy search. While much of the existing research has focused on extracting robust and secure content descriptors, a significant open challenge still remains: Most existing video hashing methods are fallible to temporal desynchronization. That is, when the query video results by deleting or inserting some frames from the reference video, most existing methods assume the positions of the deleted (or inserted) frames are either perfectly known or reliably estimated. This assumption may be okay under typical transcoding and frame-rate changes but is highly inappropriate in adversarial scenarios such as anti-piracy video search. For example, an illegal uploader will try to bypass the `piracy check' mechanism of YouTube/Dailymotion etc by performing a cleverly designed non-uniform resampling of the video. We present a new solution based on dynamic time warping (DTW), which can implement automatic synchronization and can be used together with existing video hashing methods. The second contribution of this paper is to propose a new robust feature extraction method called flow hashing (FH), based on frame averaging and optical flow descriptors. Finally, a fusion mechanism called distance boosting is proposed to combine the information extracted by DTW and FH. Experiments on real video collections show that such a hash extraction and comparison enables unprecedented robustness under both spatial and temporal attacks.

Much previous work in video fingerprinting has focused on robustness and security issues, but the compactness requirement, i.e., the hash should be of a short length with acceptable robustness and discriminability, continues to be a significant practical challenge. In this paper, we propose a video fingerprinting method with explicit attention on compactness. First, we develop a new graphical representation of the video which reduces temporal redundancies and makes robust feature extraction much more economical. Second, a randomized adaptive quantizer is proposed to further decrease the final hash length while maintaining acceptable detection performance in terms of receiver operating characteristics (ROCs). Experimental results reveal that the proposed method offers a more favorable robustness versus discriminability tradeoff over the state of the art particularly when the bit budget of the video fingerprint is low.

Applications of video fingerprinting range from traditional video retrieval and authentication to the more recent problem of anti-piracy search brought about by the emergence of video websites such as Youtube. Video fingerprints offer the potential of identifying in a robust and scalable manner - illegal or undesirable uploads of copyrighted video content. The principal challenge in video fingerprinting is to extract reduced dimensionality descriptors that can withstand incidental spatial and temporal distortions to the video while still allowing the discrimination of distinct videos. To address this fundamental problem, we propose to first represent a video as a graphical structure which can encode temporal relationships between video shots that are crucial to uniquely identifying the video. Next, we leverage ideas from graph theory, namely the normalized cuts graph partitioning method to divide the video representation into sub-graphs. Robust dimensionality reduction applied to these sub-graphs yields the final video hash/fingerprint. Experimental results in the form of receiver operating characteristic (ROC) curves on video databases acquired from YouTube reveal that the proposed video fingerprinting can enable a much more favorable robustness vs. discriminability trade-off over state-of-the art algorithms in video hashing.

Randomized embeddings of scale-invariant image features are proposed for retrieval of object-specific meta data in an augmented reality application. The method extracts scale invariant features from a query image, computes a small number of quantized random projections of these features, and sends them to a database server. The server performs a nearest neighbor search in the space of the random projections and returns meta-data corresponding to the query image. Prior work has shown that binary embeddings of image features enable efficient image retrieval. This paper generalizes the prior art by characterizing the tradeoff between the number of random projections and the number of bits used to represent each projection. The theoretical results suggest a bit allocation scheme under a total bit rate constraint: It is often advisable to spend bits on a small number of finely quantized random measurements rather than on a large number of coarsely quantized random measurements. This theoretical result is corroborated via experimental study of the above tradeoff using the ZuBuD database. The proposed scheme achieves a retrieval accuracy up to 94% while requiring the mobile device to transmit only 2.5 kB to the database server, a significant improvement over 1-bit quantization schemes reported in prior art.

The goal of video hashing is to design hash functions that summarize videos by short fingerprints or hashes. While traditional applications of video hashing lie in database searches and content authentication, the emergence of websites such as YouTube and DailyMotion poses a challenging problem of anti-piracy video search. That is, hashes or fingerprints of an original video (provided to YouTube by the content owner) must be matched against those uploaded to YouTube by users to identify instances of "illegal" or undesirable uploads. Because the uploaded videos invariably differ from the original in their digital representation (owing to incidental or malicious distortions), robust video hashes are desired. We model videos as order-3 tensors and use multilinear subspace projections, such as a reduced rank parallel factor analysis (PARAFAC) to construct video hashes. We observe that, unlike most standard descriptors of video content, tensor-based subspace projections can offer excellent robustness while effectively capturing the spatio-temporal essence of the video for discriminability. We introduce randomization in the hash function by dividing the video into (secret key based) pseudo-randomly selected overlapping sub-cubes to prevent against intentional guessing and forgery. Detection theoretic analysis of the proposed hash-based video identification is presented, where we derive analytical approximations for error probabilities. Remarkably, these theoretic error estimates closely mimic empirically observed error probability for our hash algorithm. Furthermore, experimental receiver operating characteristic (ROC) curves reveal that the proposed tensor-based video hash exhibits enhanced robustness against both spatial and temporal video distortions over state-of-the-art video hashing techniques.

The problem of summarizing videos by short fingerprints or hashes has garnered significant attention recently. While traditional applications of video hashing lie in database search and content authentication, the emergence of websites such as YouTube and DailyMotion poses a challenging problem of anti-piracy video search. That is, hashes or fingerprints of an original video (provided to YouTube by the content owner) must be matched against those uploaded to YouTube by users to identify instances of “illegal” or undesirable uploads. Because the uploaded videos invariably differ from the original in their digital representation (owing to incidental or malicious distortions), robust video hashes are desired. In this paper, we model videos as order-3 tensors and use multilinear subspace projections, such as a reduced rank parallel factor analysis (PARAFAC) to construct video hashes. We observe that unlike most standard descriptors of video content, tensor based subspace projections can offer excellent robustness while effectively capturing the spatio-temporal essence of the video for discriminability. We further randomize the construction of the hash by dividing the video into randomly selected overlapping sub-cubes to prevent against intentional guessing and forgery. The most significant gains are seen for the difficult attacks of spatial (e.g. geometric) as well as temporal (random frame dropping) desynchronization. Experimental validation is provided in the form of ROC curves and we further perform detection-theoretic analysis which closely mimics empirically observed probability of error.