Increasing resolutions push the throughput requirements of video codecs and complicate the challenges encountered during their cost-efficient implementations. We propose an FPGA implementation of a high-performance MPEG-4 simple profile video decoder, capable of parsing multiple bitstreams from different encoder sources. Its video pipeline architecture exploits the inherent functional parallelism and enables multi-stream support at a limited FPGA resource cost compared to a single stream version. The design is scalable with a number of added compile-time parameters - including maximum frame size and number of input bitstreams - which can be set by the user to suit his application.

The cost-efficient implementation of video codecs requires a set of methodologies and decision taking at different levels in the design flow. We combine upfront algorithmic tuning with memory centric optimizations to transform the video application into a system consisting of functional blocks with localized data processing and a tailored memory hierarchy. This memory optimized functional description is the leverage for the cost-efficient mapping of the system on integrated multimedia platforms. It closely reflects the real implementation constraints and consequently allows for steering the architecture selection in a correct way. The proposed approach is demonstrated on a MPEG-4 video encoder and leads to its implementation as a pipelined system. Hardware development of the motion estimation validates that the high-level memory centric concepts are applicable and realizable at the lowest level. The motion estimation kernel supports up to 30 CIF f/s with minimized processing element requirements and data input rates.

Terminal quality of service is the process of optimally scaling down the decoding and rendering computations to the available processing power, while maximizing the overall perceived quality. This process is applied in a real-time three-dimensional (3-D) decoding and rendering engine, exploiting scalable MPEG-4 coding algorithms. We derive a relation between the quality of the 3-D rendered objects and their processing requirements, expressed by simple CPU time models. The performance dependency parameters are in direct relation to the high-level 3-D content characteristics (number of triangles, number of rendered screen pixels per object) and are calibrated for the platform under test. Examples show that, for any pre-established frame rate, the platform workload can reliably be anticipated for adjusting the content and process parameters accordingly.

The realization of new MPEG-4 functionality, applicable to three-dimensional graphics texture compression and image database access over the Internet, is demonstrated on a heterogeneous platform with several unique features. First, applying our system-level design methodologies effectively removes all data transfer and storage overhead that comprises the main bottleneck in the original system description. Second, a first-of-a-kind application specific solution, called Ozone, accelerates the embedded-zero-tree based encoding and is capable of compressing 30 color CIF images per second. The entire application is running on the Ozone coupled to a PC.

One of the key functionalities in today's multimedia applications is streaming. However unlike video, streaming 3D content remains rare, especially for textures. In this paper, we present an intelligent, view-dependent approach for streaming and decoding MPEG-4 textures in order to save network and platform resources, while minimizing the perceived quality loss. Different regions of the 3D object's texture are independently decoded up to a desired quality. Whenever viewing conditions change, newly visible texture regions are transmitted and/or already transmitted regions are refined. This 3D view-dependent decoding of MPEG-4 textures is possible by exploiting the error resilience feature in the MPEG-4 texture-coding tool. It allows a high flexibility (i.e. number of possible quality levels) but at the cost of an increased bitstream size. However, since streaming is possible, the transmission of the bitstream is spread over time, reducing the instantaneous bandwidth and processing cost.

Due to the large amount of data transfers it involves, the motion estimation (ME) engine is one of the most power-consuming components of any predictive video codec. As a consequence, power-optimized video coding primarily relies on a carefully designed motion estimator. This paper first presents a block ME algorithm that meets high-quality inter-frame prediction and low computational complexity requirements. It relies on a set of rules common to all recent fast and adaptive ME algorithms, but is designed in order to allow for easy and prolific data reuse. The adjacent order of the candidate positions during the search increases the locality and maintains a near-regular data flow, which results in a decrease of the data transfers and a low control complexity. Together with the computational complexity reduction, it enables cost-efficient very large scale integration realizations. A pipelined parallel architecture is then proposed and discussed. It is generic in the sense that it is suited both to the full-pel and half-pel ME. It is efficient because it allows for close to 100% hardware utilization and a sharp decrease of the peak memory bandwidth. It is suited to low-power implementation, as it enables larger data reuse factors for the most probable stages of the adaptive algorithm, which reduces the average memory bandwidth and power consumption.

info:eu-repo/semantics/published

The realization of new MPEG-4 functionality, applicable to 3D graphics texture compression and image database access over the Internet, is demonstrated in a PC-based compression system.Applying our system-level design methodologies effectively removes all implementation bottlenecks.A first-of-a-kind ASIC, called Ozone, accelerates the Embedded Zero Tree based encoding and is capable of compressing 30 color CIF images per second.

The wavelet transform is a popular filter-based decomposition algorithm used in novel multimedia standards and applications. Due to its intrinsic use of filtering operations, the choice of an appropriate filter and the normalisation of its coefficients determines the number of bits needed to represent the transformed image. We show that depending on the normalisation of the filters, the wavelet coefficients either saturate or grow exponentially for successive transformation levels. A procedure to create images that excite worst case maxima in the transform domain is described. The bit-range expansion for natural images is computed and follows the trend of the worst case results.

This paper addresses the problem of minimizing memory size and memory accesses in multiresolution texture coding architectures for discrete cosine transform (DCT) and wavelet-based schemes used, for example, in virtual-world walk-throughs or facial animation scenes of an MPEG-4 system. The problem of minimizing the memory cost is important since memory accesses, memory bandwidth limitations, and in general the correct handling of the data flows have become the true critical issues in designing high-speed and low-power video-processing architectures and in efficiently using multimedia processors. For instance, the straightforward implementation of a multiresolution texture codec typically needs an extra memory buffer of the same size as the image to be encoded/decoded. We propose a new calculation schedule that reduces this buffer memory size with up to two orders of magnitude, while still ensuring a number of external (off-chip) memory accesses that is very close to the theoretical minimum. The analysis is generic and is therefore useful for both wavelet and multiresolution DCT codecs.

Data transfers and storage are crucial cost factors in multimedia systems. Systematic methodologies are needed to obtain dramatic reductions in terms of power, area and cycle count. Upcoming multimedia processing applications will require high memory bandwidth. In this paper, we estimate that a software reference implementation of an MPEG-4 video encoder typically requires five Gtransfers/s to main memory for a simple profile level L2. This shows a clear need for optimization and the use of intermediate memory stages. By applying our ACROPOLIS methodology, developed mainly to relieve this data access bottleneck, we have arrived at an implementation which needs a factor 65 less background accesses. In addition, we also show that we can heavily improve on the memory transfers, without sacrificing speed (even gaining about 10% on cache misses and cycles for a DEC Alpha), by aggressive source code transformations.

The upcoming multimedia processing applications will require high memory bandwidth. We estimate that a software reference implementation of an MPEG-4 video decoder typically requires 200 Mtransfers/s to memory to decode 1 CIF (352/spl times/288) video object plane (VOP) at 30 frames/s. This imposes a high penalty in terms of power but also performance. However, we also show that we can heavily improve on the memory accesses and data transfers, without sacrificing speed (even gaining about 10% on cache misses and cycles for a DEC Alpha), by aggressive code transformations. For this purpose, we have applied an extended version of our data transfer and storage exploration methodology, partly supported in the ATOMIUM environment, which was originally developed for custom hardware implementations.