In this paper we present a scalable and high performance near-duplicate image search method. The proposed algorithm follows the common paradigm of computing local features around repeatable scale invariant interest points. Unlike existing methods, much shorter hashes are used (40 bits). By leveraging on the shortness of the hashes, a novel high performance search algorithm is introduced which analyzes the reliability of each bit of a hash and performs content adaptive hash lookups by adaptively adjusting the "range" of each hash bit based on reliability. Matched features are post-processed to determine the final match results. We experimentally show that the algorithm can detect cropped, resized, print-scanned and re-encoded images and pieces from images among thousands of images. The proposed algorithm can search for a 200x200 piece of image in a database of 2,250 images with size 2400x4000 in 0.020 seconds on 2.5GHz Intel Core 2.

In this paper, we propose a method for peer assisted on demand delivery of scalable video using an optimal, distributed caching algorithm for cache constrained peers. We assume that there are one or more content delivery servers that are provisioned to support delivery of the base layer for all of the peers, and the enhancement layer is only supported for a fraction of peers. It has been shown that in such an application, peer assisted content distribution improves the delivery of the content. In this paper, we use peer assisted delivery of the enhancement layer and show that in a cache constrained video-on-demand environment, availability of the cached content becomes a bottleneck. By using peer arrival rates and supply-demand analysis of the peers, we propose better caching policies at each peer. One proposed policy (referred to as "static optimal") optimizes the utility of the currently available caching opportunities. The other proposed policy (referred to as "dynamic programming" solution) on the other hand performs utility optimization by further considering the future caching opportunities during the streaming operation. Experimental results show that the proposed methods allow more peers to access the enhancement layer; hence, more peers experience improved video quality compared to traditional caching methods.

In this paper, we consider a hybrid P2P video on-demand architecture that utilizes both the server and the peer resources for efficient transmission of popular videos. In our system architecture, each peer dedicates some cache space to store a particular segment of a video file as well as some of its upload bandwidth to serve the cached segment to other peers. Peers join the system and issue a streaming request to a control server. Control server directs the peers to streaming servers or to other peers who have the desired video segments. Control server also decides which peer should cache which video segment. Our main contribution in this paper is to determine the proper caching strategies at peers such that we minimize the average load on the streaming servers.

We propose a block-based transform optimization and associated image compression technique that exploits regularity along directional image singularities. Unlike established work, directionality comes about as a byproduct of the proposed optimization rather than a built in constraint. Our work classifies image blocks and uses transforms that are optimal for each class, thereby decomposing image information into classification and transform coefficient information. The transforms are optimized using a set of training images. Our algebraic framework allows straightforward extension to non-block transforms, allowing us to also design sparse lapped transforms that exploit geometric regularity. We use an EZW/SPIHT like entropy coder to encode the transform coefficients to show that our block and lapped designs have competitive rate-distortion performance. Our work can be seen as nonlinear approximation optimized transform coding of images subject to structural constraints on transform basis functions.

In this paper, we propose a novel method for watermarking compressed video for streaming. The proposed method performs motion compensated watermarking. There are two consequences of this; first we see that the impact of the watermark on the final bitrate is negligible, and the proposed method has inherent robustness to mild packet losses in the channel. Experimental results show that the proposed method operates at lower distortion levels and has higher watermark detection rates compared to current watermarking methods. We observe 20% bitrate reduction for the same watermark detection rate. We also show that when data partitioning is used over a packet loss channel the proposed method performs at acceptable watermark detection rates.

This paper proposes a complete stochastic framework for RD optimal encoder design for video over error-prone networks, which applies to any motion-compensated predictive video codec. The distortion measure has been taken as the mean square error over an ensemble of channels given an estimate of the instantaneous packet loss probability. We show that 1) the optimal motion compensated prediction, in the MSE sense, requires computation of the expected value of the reference frames, and 2) calculation of the MSE (distortion measure) requires computation of the second moment of the reference frames. We propose a recursive procedure for the computation of both the expected value and second moment of the reference frames, which are together called the stochastic frame buffer. Furthermore, we propose a stochastic RD optimization method for selection of the optimal macroblock mode and motion vectors given the instantaneous packet loss probability. If available, channel feedback can also be incorporated into the proposed stochastic framework. However, the proposed framework does not require a feedback channel to exist, and when it exists, it does not have to be lossless. In the absence of any packet losses, the proposed stochastic framework reduces to the well-known deterministic RD optimization procedures. One possible application of the optimal stochastic framework would be for multicast streaming to an ensemble of receivers. Experimental results indicate that the proposed framework outperforms other available error tracking and control schemes.

We propose new models and methods for rate-distortion (RD) optimal video delivery over IP, when packets with bit errors are also delivered. In particular, we propose RD optimal methods for slicing and unequal error protection (UEP) of packets over IP allowing transmission of packets with bit errors. The proposed framework can be employed in a classical independent-layer transport model for optimal slicing, as well as in a cross-layer transport model for optimal slicing and UEP, where the forward error correction (FEC) coding is performed at the link layer, but the application controls the FEC code rate with the constraint that a given IP packet is subject to constant channel protection. The proposed method uses a novel dynamic programming approach to determine the optimal slicing and UEP configuration for each video frame in a practical manner, that is compliant with the AVC/H.264 standard. We also propose new rate and distortion estimation techniques at the encoder side in order to efficiently evaluate the objective function for a slice configuration. The cross-layer formulation option effectively determines which regions of a frame should be protected better; hence, it can be considered as a spatial UEP scheme. We successfully demonstrate, by means of experimental results, that each component of the proposed system provides significant gains, up to 2.0 dB, compared to competitive methods.

We consider the problem of blind watermark detection from an image, when the image has undergone a geometric attack. Existing schemes for solving this problem involve insertion of periodic marks or templates, invariant properties of transforms, and feature point based techniques. A major deficiency of these approaches is either a security leak (owing to redundancy in the mark) and/or poor watermark detection performance esp. under lossy geometric attacks, e.g. cropping. In this paper, we introduce a novel watermark synchronization paradigm by using perceptual image hashes. We exploit the fact that a perceptual hash acts as a robust as well as randomized digest of image content, and use it to embed a synchronization mark into the image. Unlike existing synchronization schemes, our synchronization mark has favorable security and robustness to adversarial attacks and hence, cannot be easily removed. Experimental results confirm that our proposed algorithm has significantly lower watermark detection error probabilities than existing methods.

In this paper we propose a method for video delivery over bit error channels. In particular, we propose a rate distortion optimal method for slicing and unequal error protection (UEP) of packets over bit error channels. The proposed method performs full frame based search using a novel dynamic programming approach to determine the optimal slicing configuration in a practically short time. Also we propose a rate and distortion estimation technique that decreases the time to evaluate the objective function for a slice configuration. The proposed method can perform rate-distortion UEP that can be used over forward error correction (FEC) capable channels. We show that the proposed method successfully exploit the local dynamics of a video frame and perform more than 1 dB better than common methods.

In this paper, we introduce a method for video transmission over lossy channels. The proposed method is similar to NEWPRED and uses feedback information to stop error propagation. Main operational difference is replication of packet loss and error concealment process at the encoder and using the possibly error concealed frames as reference frames in the motion prediction loop. We have studied the optimization of the proposed method under moderate error rates where ACK/NACK mode NEWPRED does not efficiently solve the problem. The proposed solution is an adaptive method that optimizes the operation based on the expected error propagation. We present comparison results with NEWPRED under various channel conditions and video sources.

In this paper we propose an error control scheme for video communications over lossy channels. The proposed algorithm uses stochastic frame buffers (SFB) to determine the expected decoder error by using channel simulated frames during motion vector (MV) and reference frame search, macroblock (MB) mode decisions and residual selection. Unlike previous methods, our algorithm also focuses on optimizing the MV and prediction frame selection and residual selection over error prone channels. The algorithm minimizes a Lagrangian to achieve the optimal result for mode and MVs. If there is channel feedback, the system can utilize it to update the SFB. We perform experiments that compare our system with intra refreshes, an improved NEWPRED and optimal mode switching algorithms such as ROPE under various conditions such as existence of a feedback channel or not. We show that the proposed system outperforms others.

In this paper, we study the data partitioning (DP) and its optimization for H264 video coding standard. H264 does not include DP in baseline profile, which is the most suitable profile for low delay, low complexity, and loss prone environments. To analyze the optimization of DP, we first introduce the concept of subchannels to abstract the physical layer. This allows us to move channel coding from application layer to physical layer. Then, we build the video encoder system around NEWPRED (1996) so that error propagation and its analysis is eliminated. Finally, we provide macroblock and slice level optimizations that result in optimal mode decisions and unequal error protection (LTEP) rates for data partitions. Experimental results show about 0.5 dB performance increase as compared to no data partitioning.