Video streaming on mobile devices such as PDA's, laptop PCs, pocket PCs and cell phones is becoming increasingly popular. These mobile devices are typically constrained by their battery capacity, bandwidth, screen resolution and video decoding and rendering capabilities. Consequently, video personal

We present a scalable, lightweight, no-reference framework to infer video QoE. Our framework revolves around a one time offline construction of a k-dimensional space, which we call the QoE space. The k-dimensions accommodate k parameters (network-dependent/independent) that potentially affect video quality. The k-dimensional space is partitioned to N representative zones, each with a QoE index. Instantaneous parameter values are matched with the indices to infer QoE. To validate our framework, we construct a 3-dimensional QoE space with bit-rate, loss, and delay as the principal components. We create 18 video samples with unique combinations of the 3 parameters. 77 human subjects rated these video samples on a scale of 1 to 5 to create the QoE space. In a second set of survey, our predicted MOS was compared to 49 human responses. Results show that our MOS predictions are in close agreement with subjective perceptions. An implementation of our framework on standard Linux PC shows we can compute 20 MOS calculations per second with 3 parameters and 18 partitions of the QoE space.

Digital video playback on mobile devices is fast becoming widespread and popular. Since mobile devices are typically resource constrained in terms of network bandwidth, battery power and available screen resolution, it is often necessary to formulate special encoding techniques in order to optimize power consumption during video streaming and playback. The existing H.264 standard is popular for video encoding on mobile devices, since it results in a low-bitrate video with visual clarity that is adequate for video playback on mobile devices. However, due to the complexity of the H.264 representation, the video decoding procedure is typically computationally intensive. In this paper, we propose a novel lossy video representation termed as Ligne-Claire (LC) video. LC videos are obtained via graphics overlay of outlines or silhouettes of objects in the video over an approximated texture video. Since the playback of LC video is typically meant for mobile devices, the visual quality of video is adequate for most mobile applications wherein the semantic content of the video can be characterized by object shapes and approximate texture information. Experimental results presented in the paper demonstrate that the proposed lossy LC video encoding scheme results in power savings of 50% or more during video playback compared to standard H.264-encoded videos, of similar video file size. In order to evaluate the visual quality of the LC video, we compare the performance of LC videos with H.264-encoded videos in the context of some typical computer vision tasks. Our results indicate that the performance of the computer vision algorithms on these videos is similar. This fact, coupled with subjective evaluation, and the resulting significant power savings, indicates that the proposed LC representation can be used effectively to encode video for power-constrained mobile devices.

Progressive download of multimedia objects over the Internet (e.g. www.youtube.com), where the video is downloaded and viewed during the download process, has become an increasingly popular alternative to multimedia streaming. Due to the fluctuating bandwidth and latency of the Internet, progressive download is often not fast enough, often resulting in intermittent stalling of the video. In this paper, we first propose a variation of the existing MPEG Fine Grained Scalability (FGS) profile to create a layered video representation that is suitable for progressive download in an environment characterized by varying bitrate. We also propose an efficient caching scheme that is specifically tailored for the proposed layered video representation. The proposed layered version of the Greedy-Dual-Size cache replacement policy is shown to reduce the latency observed by the client during progressive download of video in a varying bitrate environment. Experimental results demonstrate that the proposed caching scheme improves the latency of progressive video downloads as well as the server efficiency.

MPEG-4 body animation parameters (BAP) are used for animation of MPEG-4 compliant virtual human-like characters. Distributed virtual reality applications and networked games on mobile computers require access to locally stored or streamed compressed BAP data. Existing MPEG-4 BAP compression techniques are inefficient for streaming, or storing, BAP data on mobile computers, because: 1) MPEG-4 compressed BAP data entails a significant number of CPU cycles, hence significant, unacceptable power consumption, for the purpose of decompression, 2) the lossy MPEG-4 technique of frame dropping to reduce network throughput during streaming leads to unacceptable animation degradation, and 3) lossy MPEG-4 compression does not exploit structural information in the virtual human model. In this article, we propose two novel algorithms for lossy compression of BAP data, termed as BAP-Indexing and BAP-Sparsing. We demonstrate how an efficient combination of the two algorithms results in a lower network bandwidth requirement and reduced power for data decompression at the client end when compared to MPEG-4 compression. The algorithm exploits the structural information in the virtual human model, thus maintaining visually acceptable quality of the resulting animation upon decompression. Consequently, the hybrid algorithm for BAP data compression is ideal for streaming of motion animation data to power- and network-constrained mobile computers

Human Motion Capture (MoCap) data can be used for animation of virtual human-like characters in distributed virtual reality applications and networked games. MoCap data compressed using the standard MPEG-4 encoding pipeline comprising of predictive encoding (and/or DCT decorrelation), quantization, and arithmetic/Huffman encoding, entails significant power consumption for the purpose of decompression. In this paper, we propose a novel algorithm for compression of MoCap data, which is based on smart indexing of the MoCap data by exploiting structural information derived from the skeletal virtual human model. The indexing algorithm can be fine-controlled using three predefined quality control parameters (QCPs). We demonstrate how an efficient combination of the three QCPs results in a lower network bandwidth requirement and reduced power consumption for data decompression at the client end when compared to standard MPEG-4 compression. Since the proposed algorithm exploits structural information derived from the skeletal virtual human model, it is observed to result in virtual human animation of visually acceptable quality upon decompression.

Multi-resolution quad-tree based image representations are useful for image and video encoding at varying bit rates. Existing algorithms use a difference measure of color values to trim branches of the quad-tree, which results in the various spatial regions of the image being represented at different resolutions. However, color difference-based quad-tree branch pruning often leads to sub-optimal multi-resolution images, where interesting features of the image are represented at low resolution, and/or uninteresting image regions are represented at high resolution. In this paper, we present a mask-based method for pruning a quadtree representation of a multi-resolution image. The masks highlight various features of the image which require the best resolution, allowing tightly controlled pruning of the quad-tree branches. We suggest three main groups of masks, which are suitable for most types of applications requiring multi-resolution images and videos.

The MPEG-4 Fine Grained Scalability (FGS) profile aims at scalable video encoding, in order to ensure efficient video streaming in networks with fluctuating bandwidth. In order to allow very low bit rate streaming, the Base Layer of an FGS video is encoded at a very low bit rate, resulting in very low video quality. In this paper, we propose FGS-MR, which uses content aware multi-resolution video frames to obtain better video quality for a target bit rate, compared to existing MPEG-4 FGS Base Layer video encoding schemes. FGS-MR is an integrated approach that requires only encoder side modification, and is transparent to the decoder. In addition, FGS-MR can be used with any existing MPEG-4 codec which supports FGS, since it entails smart video preprocessing and does not involve any components from the MPEG-4 compression pipeline. FGS-MR is a mask based technique. We have demonstrated an unsupervised algorithm to automatically create the mask from a given video sequence.

Animation of Virtual Humans (avatars) is done typically using motion data files that are stored on a client or streaming motion data from a server. Several modern applications require avatar animation in mobile networked virtual environments comprising of power constrained clients such as PDAs, Pocket-PCs and notebook PCs operating in battery mode. These applications call for efficient compression of the motion animation data in order to conserve network bandwidth, and save power at the client side during data reception and motion data reconstruction from the compressed file. In this paper, we have proposed and implemented a novel file format, termed the Quantized Motion Data (QMD) format, which enables significant, though lossy, compression of the motion data. The motion distortion resulting from the reconstructed motion from the QMD file is minimized by intelligent use of the hierarchical structure of the skeletal avatar model. The compression gained by using the QMD files for the motion data is more than twice achieved via standard MPEG-4 compression using a pipeline comprising of quantization, predictive encoding and arithmetic coding. In addition, considerably fewer CPU cycles are needed to reconstruct the motion data from the QMD files compared to motion data compressed using the MPEG-4 standard.