This paper is concerned with the generation, acquisition, standardized representation and transport of video metadata. The use of MPEG standards in the design and development of interoperable media architectures and web services is discussed. A high-level discussion of several algorithms for metadata extraction is presented. Some architectural and algorithmic issues encountered when designing services for real-time processing of video streams, as opposed to traditional offline media processing, are addressed. A prototype real-time video analysis system for generating MPEG-7 Audiovisual Description Profile from MPEG-2 transport stream encapsulated video is presented. Such a capability can enable a range of new services such as content-based personalization of live broadcasts given that the MPEG-7 based data models fit in well with specifications for advanced television services such as TV-Anytime and Alliance for Telecommunications Industry Solutions IPTV Interoperability Forum.

In large-scale IPTV systems, it is essential to maintain high service quality while providing a wider variety of service features than typical traditional TV. Thus service quality assessment systems are of paramount importance as they monitor the user-perceived service quality and alert when issues occurs. For IPTV systems, however, there is no simple metric to represent user-perceived service quality and Quality of Experience (QoE). Moreover, there is only limited user feedback, often in the form of noisy and delayed customer calls. Therefore, we aim to approximate the QoE through a selected set of performance indicators in a proactive (i.e., detect issues before customers reports to call centers) and scalable fashion.

In this paper, we describe a novel architecture for content and delivery validation for IPTV systems. The system generates multimedia events derived from IPTV content analysis and its delivery. Events are aggregated, filtered and, through a rules-based system, processed and prioritized to provide monitoring and alerting capabilities. Offline event processing along with a video archiving capability allows for detailed forensic analysis. The novelty of the approach includes the correlation and the combination of content-specific and network-specific events in a normalized representation to allow for complex forensic monitoring and alerting. Applications include content security assurance and metadata verification.

In this paper we propose a novel approach to find a global correspondence between two images by maximizing mutual information in the presence of large scale changes and rotations. Our approach combines the local descriptors with global search. We have tested our method on various test images and compared the matching performance with the SIFT descriptors. The experimental results show that the proposed local descriptor and graph-based search provides robust point matching for scale, rotation and illumination changes

In this paper, a configurable coding scheme is proposed and analyzed with respect to computational complexity and distortion (C-D). The major coding modules are analyzed in terms of computational C-D in the H.263 video coding framework. Based on the analyzed data, operational C-D curves are obtained through an exhaustive search, and the Lagrangian multiplier method. The proposed scheme satisfies the given computational constraint independently of the changing properties of the input video sequence. A technique to adaptively control the optimal encoding mode is also proposed. The performance of the proposed technique is compared with a fixed scheme where parameters are determined by off-line processing. Experimental results demonstrate that the adaptive approach leads to computation reductions of up to 19%, which are obtained with test video sequences and compared to the fixed, while the peak signal-to-noise ratio degradations of the reconstructed video are less than 0.05 dB.

Traditional scalable video encoders sacrifice coding efficiency to reduce error propagation because they have avoided using enhancement-layer (EL) information to predict the base layer (BL) to prevent the error propagation termed "drift". Drift can produce very poor video quality if left unchecked. We propose a video coder with significantly better compression efficiency because it intentionally allows the drift produced by predicting the BL from the EL. Our drift management system balances the tradeoff between compression efficiency and error propagation. The proposed scalable coder uses a spatially adaptive procedure that optimally selects key encoder parameters: the quantizer and the prediction strategy. Our numerical results indicate the encoder is very powerful, and the selection procedure is effective. The video quality of our coder at low rates is only marginally worse than the drift-free case, while its overall compression efficiency is not much worse than a one-layer nonscalable encoder.

In this paper we discuss a simple method to improve the lighting conditions of a scene or video sequence. We focus on modifying the intensities of existing light sources and on inserting additional virtual lights into a scene. We target video conferencing applications where typical lighting conditions (i e the average office or home environment) are poor. Our model is much simpler and is a departure from approaches such as re-lighting, developed in computer-augmented reality (CAR) where the scene is first reconstructed geometrically by means of computer vision techniques We rely on a well-known illumination equation that takes into account light attenuation, plus lambertian and specular reflection Our first scenario models a 3D space in which the image or video frame lies on a 2D plane. Virtual lights are placed in the 3D space and illuminate the 2D image We then propose a refined model for talking head video sequences where the talking head is detected, tracked and its surface is modeled with a 3D ellipsoid. Results show that convincing lighting improvements can be achieved in both scenarios.

Scalable video coders have traditionally avoided using enhancement-layer (EL) information to predict the base layer (BL), so as to avoid so-called "drift". As a result, they are less efficient than a one-layer coder. Fine granularity scalable (FGS) coders avoid using EL information to predict the EL as well, suffering even further inefficiencies. In this paper, we explore a scalable video coder that allows drift, by predicting the BL from EL information. However, we show that through careful management of the amount of drift introduced, the video quality at low rates is only marginally worse than the drift-free case, while the overall compression efficiency is not much worse than a one-layer encoder.

Different communication media create different shopping experiences, e.g., the phone vs. the web. In this study, we examined the early formation of trust and the likelihood that a shopper will return to a website for subsequent purchases. Consumers were asked to select a blender as a gift for someone they knew using one of four types of user interfaces: (1) a standard web interface with no audio or real time interpersonal communication, (2) a web interface with text and audio descriptions of the products, (3) a web interface augmented with Instant Messaging and the ability of a sales agent to push web pages to the customer's web browser, and (4) a TV-based application that allowed a salesperson to talk to the customer while pushing catalog items to the customer's TV. Results indicate that real-time interactivity, but not voice, increased judgments of friendliness and of the trustworthiness of the salesperson. In this paper we discuss these and other results, and their implications for e-commerce.

Communication and commerce by web or phone creates benefits and challenges for both buyer and seller. Websites provide convenience and visualization; telephones provide voice and real-time interaction. To combine key elements of these experiences, we developed PhoneChannel. Using PhoneChannel, a PC user while talking on the telephone can display visuals on the other person's television. How do these different media affect the consumer experience? In a recent laboratory study, prospective homebuyers selected houses of interest using web, telephone, or PhoneChannel. Using the telephone or PhoneChannel led to higher trust; but using web or PhoneChannel led to higher ratings on convenience, enjoyment, and 'good method' scales.

In this paper we propose a novel bit allocation scheme for MPEG-2 constant bit rate (CBR) video coding using a model of the early stages of human vision. On the basis of this model we derive a measure of the macroblock activity different from the one proposed in the MPEG-2 test model 5 (TM5). Experiments, obtained by substituting the proposed perceptual activity measure to the one proposed by the MPEG-2 TM5, yield better results, among which are the improvement of the perceptual quality for a fixed bitrate and vice-versa.

The purpose of this paper is twofold first to present two other variants of the algorithm shown by Basso, Geurtz, and Kunt (see SPIE-VCIP 93, volume SPIE 2094-I, p.976-986, Boston, USA, Nov. 8-11, 1993) that we call data restoring and error distribution algorithms. Second to discuss a new coding philosophy, based on modelization of the sign of 2D bandpass signals. The data restoring algorithm improves the reconstruction quality and convergence speed while the error distribution one can convergence faster with an acceptable reconstruction quality. In the proposed system a bandpass 2D signal is viewed as a baseband signal that is modulating a carrier. The great part of the information relative to the carrier component is known a priori. Furthermore it constitutes a major part of the sign information of the 2D signal. The method models the sign of carrier component and subtracts it from the sign of the original signal. This technique is applied in a subband based still image coding system. The image is decomposed in subbands that are modeled in terms of the carrier and modulation components. For each subband the sign of logical XOR between a carrier refined model and the original subband is coded, together with the carrier model parameters and refinement parameters. Interesting coding results have been obtained.>

In this paper we propose a complete subband-based codec for image compression purposes, using a new 2D algorithm based on zero-crossing and energy information. Although signal representation based on sign information is attractive from a coding point of view, two major problems inhibited its practical use, namely the large amount of side information needed for obtaining a stable reconstruction and the efficient treatment of inherently 2D signals. The new algorithm significantly improves the reconstruction results obtained in our previous work, thanks to its improved convergence and its capabilities to deal with the two-dimensional nature of the subband image signals. The zero-crossing information is compressed in a lossless way using the JBIG standard and the energy information in a lossy way using an arithmetic coder. Good results have been obtained both at the reconstruction and at the coding level, which show the feasibility of the global scheme.