MPEG-7 has ignited interest in industry with broad interests in media, content management, content provision, broadcasting, computers, telecommunications, as well as researchers and academia. Overall, the MPEG-7 standard is currently in a fairly mature stage and is on its way to final standardization later this year. The pace of progress in development of MPEG-7 has been quite frantic. This special issue provides snapshot of the status of MPEG-7 technology as of around April 2001. This special issue contains only invited papers from experts who are involved in MPEG-7 standardization. It is designed to be useful to beginners as well as experts alike as it covers various topics at different depths.

MPEG-7, formally known as the Multimedia Content Description Interface, includes standardized tools (descriptors, description schemes, and language) enabling structural, detailed descriptions of audio-visual information at different granularity levels (region, image, video segment, collection) and in different areas (content description, management, organization, navigation, and user interaction). It aims to support and facilitate a wide range of applications, such as media portals, content broadcasting, and ubiquitous multimedia. We present a high-level overview of the MPEG-7 standard. We first discuss the scope, basic terminology, and potential applications. Next, we discuss the constituent components. Then, we compare the relationship with other standards to highlight its capabilities.

In this paper, we describe description schemes (DSs) for image, video, multimedia, home media, and archive content proposed to the MPEG-7 standard. MPEG-7 aims to create a multimedia content description standard in order to facilitate various multimedia searching and filtering applications. During the design process, special care was taken to provide simple but powerful structures that represent generic multimedia data. We use the extensible markup language (XML) to illustrate and exemplify the proposed DSs because of its interoperability and flexibility advantages. The main components of the image, video, and multimedia description schemes are object, feature classification, object hierarchy, entity-relation graph, code downloading, multi-abstraction levels, and modality transcoding. The home media description instantiates the former DSs proposing the 6-W semantic features for objects, and 1-P physical and 6-W semantic object hierarchies. The archive description scheme aims to describe collections of multimedia documents, whereas the former DSs only aim at individual multimedia documents. In the archive description scheme, the content of an archive is represented using multiple hierarchies of clusters, which may be related by entity-relation graphs. The hierarchy is a specific case of entity-relation graph using a containment relation. We explicitly include the hierarchy structure in our DSs because it is a natural way of defining composite objects, a more efficient structure for retrieval, and the representation structure used in MPEG-4. We demonstrate the feasibility and the efficiency of our description schemes by presenting applications that already use the proposed structures or will greatly benefit from their use. These applications are the visual apprentice, the AMOS-search system, a multimedia broadcast news browser, a storytelling system, and an image meta-search engine, MetaSEEk.

We first present new concepts applicable to the design of multimedia search and retrieval schemes in general, and to MPEG-7 in particular, the multimedia description standard in progress. Raw multimedia data is assumed to exist in the form of programs that typically consist of a combination of media types such as visual, audio, and text. We partition each such media stream into smaller units based on actual physical events. These physical events within each media stream can then be effectively indexed for retrieval. The concept of logical events is introduced next; we define logical events as those that can provide different "views" of the content as may be desired by a user. Such events usually result from either the correlation of events that cross different media types, of by merging recursively chosen events from a lower level within each media type. We then address the related issue of how to develop a practical multimedia information retrieval system that exploits the aforementioned concepts of physical and logical events as well as other aspects such as storage, representation and indexing to enable efficient search, retrieval, and browsing. Finally, we implement the proposed concepts and solutions within a multimedia system that addresses a real application, effective browsing of broadcast news, and evaluate its performance.

First, a brief overview of the MPEG-4 standard is presented to highlight the fact that in order to achieve object based functionality, it requires arbitrary shaped visual objects. Next, a number of applications and their requirements are reviewed to determine the need for more than one class of solutions for visual shape representation. Then, a brief review of the solution for shape representation in MPEG-4 version 1 is presented, and, a solution that we have proposed for MPEG-4 version 2 is outlined. Details of our implicit shape coding which uses chromakey in MPEG4 systems scene description (BIFS) are then presented. Implementation of our shape representation in a real-time MPEC-4 player is then discussed. Finally we address the context in which our shape representation can address unique functionalities.

The ISO MPEG Committee, after successful completion of the MPEG-1 and the MPEG-2 standards, has recently completed the Committee Draft for MPEG-4, its third standard. MPEG-4 is designed to be an object-based standard for multimedia coding. The visual part of the standard specifies coding of both natural and synthetic video. The MPEG-4 visual standard supports coding of natural video not only in a conventional manner (using frames) but also as a collection of arbitrary shape objects (using video object planes). Further, it supports functionalities such as spatial and temporal scalability, both conventional and with arbitrary shape objects. It also supports error resilient coding for delivery of coded video on error prone channels. MPEG-4 visual standard also supports coding of synthetic video which includes still texture maps used in 3D graphics models, mesh geometry for object animation, and parameters for facial animation.

Discusses coding standards for still images and motion video. We first briefly discuss standards already in use, including: Group 3 and Group 4 for bilevel fax images; JPEG for still color images; and H.261, H.263, MPEG-1, and MPEG-2 for motion video. We then cover newly emerging standards such as JBIG1 and JBIG2 for bilevel fax images, JPEG-2000 for still color images, and H.263+ and MPEG-4 for motion video. Finally, we describe some directions beyond the standards such as hybrid coding of graphics/photo images, MPEG-7 for multimedia metadata, and possible new technologies.

Methods for object-based compression and composition of natural and synthetic video content are currently emerging in standards such as MPEG-4 and VRML. This paper describes novel techniques for compression of 2-D triangular mesh geometry and motion, enabling efficient representation and manipulation of video content. Specifically, mesh geometry is compressed by predictive coding of mesh node locations. Mesh node motion vectors are compressed by predictive techniques as well. Preliminary results show that the mesh data can be coded at a fraction of the bits used to code a typical video object.

Scalable video coding is important in a number of applications where video needs to be decoded and displayed at a variety of spatial and temporal resolution scales or when resilience to errors is necessary. In this paper, we investigate a temporal-domain approach for resolution scalable video coding. This approach was originally proposed to and since then has been included in the Moving Picture Experts Group (MPEG-2) standard. Temporal scalability, although not limited to, is particularly useful for applications such as, HDTV as well as for high quality telecommunication applications that may require high frame rate (e.g., 60 Hz) progressive video. Furthermore, it allows flexibility in use of progressive or interlaced video formats for coding even though the source video may be progressive with high frame rate. Our technique builds on the motion-compensated DCT framework of nonscalable (single layer) MPEG-2 video coding and adds motion compensated inter-layer coding. In our simulations we focus on the 2- layer temporally scalable coding and compare the performance of using progressive video format for both layers to that when interlaced video format is used for both layers; all interim video formats used are derived from original high temporal resolution progressive video. In each case we compare the performance of two inter-layer prediction configurations and evaluate their prediction efficiency.>

The second phase of the Moving Picture Experts Group (MPEG-2) is ongoing and so far has resulted in completion of algorithmic tools for video coding with this standard. This standard is generic in nature and is organized around the concept of profiles and levels to address a wide range of applications and bit-rates. It not only supports coding of interlaced and progressive format video as a single layer, but also supports scalable video coding. The MPEG-2 standard, like the MPEG-1 standard, specifies a syntax for the bitstream and the semantics of the decoding process; encoders can be designed for best tradeoff of performance versus complexity, based on the application. We describe the various tools available in the MPEG-2 standard and discuss the potential of each tool for improving coding performance or enabling new applications.

Scalable video coding is important in a number of applications where video needs to be decoded and displayed at a variety of resolution scales. In this paper, we investigate the spatial-domain approach for spatial resolution scalability. We employ the framework proposed for the ongoing second phase of Moving Picture Experts Group (MPEG-2) standard to introduce a general coding structure and discuss key issues in progressive-to-interlace and interlace-to-interlace scalable coding. We discuss the technique of spatio-temporal weighted prediction and show detailed encoder and decoder structures employing this technique. In our simulations we focus on 2-layer progressive-to-interlace spatially scalable coding and compare its performance with the simulcast technique at different bit-rates. Next, we evaluate the performance of spatially scalable coding for different picture types. Finally, we examine the efficiency of a chosen set of weight codes in spatio-temporal weighted prediction and propose improvements.

The authors address the problem of adapting the Motion Picture Experts Group (MPEG) quantizer for scenes of different complexity (at bit rates around 1 Mb/s), such that the perceptual quality of the reconstructed video is optimized. Adaptive quantisation techniques conforming to the MPEG syntax can significantly improve the performance of the encoder. The authors concentrate on a one-pass causal scheme to limit the complexity of the encoder. The system employs prestored models for perceptual quality and a bit rate that have been experimentally derived. A framework is provided for determining these models as well as adapting them to locally varying scene characteristics. The variance of an 8*8 (luminance) block is basic to the techniques developed. Following standard practice, it is defined as the average of the square of the deviations of the pixels in the block from the mean pixel value.>

This paper considers the topic of video compression at bit-rates around 1 Mbps for digital storage/playback applications. The backbone algorithm employed is the well-known motion-compensated prediction-error coder. This approach is considerably enhanced by the incorporation of conditional motion-compensated interpolation (CMCI), where some of the video frames are encoded with prediction-error coding and the remainder are encoded with motion-compensated interpolation and interpolation-error coding. The advantage of this combined technique is that interpolation errors can be coded with much fewer bits than prediction errors for similar reconstructed-picture quality. We consider two different techniques for estimating motion in CMCI- coded frames. In order to facilitate direct interaction with the bit-stream, the input sequence is divided into groups such that one frame in each group is intra-coded. The arrangement of predictive- and CMCI-coded frames can be varied; a study of these arrangements is the major focus of this paper. Structures with one-, two- and four-frame interpolation are compared. Our results indicate that one-frame interpolation is inferior to two-frame interpolation on complex test material. Four-frame interpolation is comparable in performance to two-frame interpolation as long as the motion is compensable.

A study of predictive coding of component TV signals for compression to transmit at rates that give good picture quality is reported. Designing predictors for minimum mean square prediction error is effective, both for luminance and chrominance signals. Predictors designed in this manner were found to be as good as or better than all other predictors tested. Quantizers designed for minimum mean square system error were primarily used in this study. A programmable testbed was built to permit quick comparisons. Predictors were compared on the testbed using a 2-b quantizer in order to have adequate system error to display visible differences. For simple intrafield coding, component signals compressed for transmission at 30 Mb/s gave about equivalent quality to an NTSC composite signal coded directly for transmission at 45 Mb/s. With 45-Mb/s transmission, component signal coding showed no detectable degradation on close inspection of real pictures. Maximum chrominance changes on test patterns yield visually detectable chrominance slope overload.>

We present an efficient search technique which minimizes the computations necessary for estimating the motion in video-sequences by the block matching method. We also discuss the theoretical basis for conducting such a reduced search by our technique. We then present two algorithms which employ the proposed technique for estimating the motion typical of video-conferencing environment. Next, the results of computer simulations on a real video-sequence are included which demonstrate the effectiveness of the proposed technique. Finally, the results of a study of statistical properties of block motion-compensated frame difference signals are also summarized, to assist in future choice of a coding strategy for such signals.