A new face recognition algorithm is introduced that is based on finding an optimal mapping between two images and measuring the properties of the mapping to determine the similarity between the two images. This new method is shown to be extremely robust to variations in pose, camera position and lighting for the problem of face recognition especially in an uncontrolled environment such as video surveillance. The technique does not depend on locating or training on facial features and can be applied in general to any object recognition problem. We present recognition results on current databases available to the research community and will demonstrate a real-time version of the software for access control and surveillance applications at the conference.

Transmission of digital video signals over wireless networks demands efficient compression algorithms as well as reliable coding strategies. In this project, we provide a thorough evaluation of the joint source-channel video coding methodology from two points of view: source coding design for error resilience and channel coding for error detection and recovery. We investigate the current ITU video compression standard, H.263+, for 3G wireless transmission. In particular, we concentrate on error resilient features provided within the standard and forward error correction (FEC) to find the optimal combination of various system parameters under different lossy channel conditions.

Scalability has become an important feature for video coding algorithm design for heterogeneous networks due to variable bandwidth, variable wired and wireless network conditions and variable terminal capabilities. However, the best video compression algorithms are based on temporal prediction through motion compensation which do not lend themselves naturally to a scalable framework. Hybrid coding schemes are being introduced that combine a base-layer motion-compensation (MC) coder with an enhancement layer which offers fine-grain scalability through an embedded or progressive coder. Such a framework usually results in some compression efficiency loss over the best single layer motion compensated scheme Here, we introduce a hybrid coding scheme which combines a base-layer MC coder and a finely scalable enhancement layer where the information from the base-layer is used to determine the location of the high energy signal in the enhancement layer. This coder provides better compression results than embedded approaches which do not rely on base layer information.

We propose a motion-compensated video coding system employing dense motion fields. The dense motion field is calculated at the transmitter, and the motion information is efficiently encoded and transmitted along with the residual frame. The motion estimation is performed by existing techniques in the literature, while we focus on the coding of the motion field and the displaced frame difference (DFD) frame. The dense motion field formulation leads to several novel and distinct advantages. The motion field is encoded in a lossy manner to make the motion rate manageable. The more accurate and precise motion description allows us to predict where the DFD energy will be significant, thus leading to a more efficient DFD encoder compared to applying traditional still-image coding techniques. Furthermore, the dense motion field framework allows us to refine and tailor the motion estimation process such that the resulting DFD frame is easier to encode. Simulations demonstrate superior performance against standard block-based coders, with greater advantages for sequences with more complex motion.

We presents a method for encoding the displaced frame difference (DFD) that rounds out our continuing study on video compression using dense motion fields. The main idea of the DFD coding strategy is to utilize the more dense and thus accurate motion field to predict where the sparse but significant DFD energy is located. This leads to a more efficient DFD encoder compared to applying traditional still-image coding techniques. Furthermore, the dense motion field framework allows us to refine and tailor the motion estimation process such that the resulting DFD frame is easier to encode. Simulations demonstrate superior performance against standard block-based coders, with greater advantages for sequences with more complex motion.

In this paper, we introduce an unequal error protection technique for foveation-based error resilience over highly error-prone mobile networks. For point-to-point visual communications, visual quality can be significantly increased by using foveation-based error resilience where each frame is divided into foveated and background layers according to the gaze direction of the human eye, and two bitstreams are generated. In an effort to increase the source throughput of the foveated layer, we employ unequal delay-constrained ARQ and RCPC (rate compatible punctured convolutional) codes in H.223 Annex C. In the simulation, the visual quality is increased in the range of 0.3 dB to 1 dB over channel SNR 5 dB to 15 dB.

A progressive video codec can be designed to be finely scalable, that is, it can produce one fully embedded bitstream which can be decoded at any intermediate rate up to the full target rate, producing progressively better video quality with increased bit-rate. Video scalability is an attractive feature that not only provides bit-rate adaptability to different bandwidth conditions over a heterogeneous network but also provides a framework for introducing graceful degradation in video quality in the presence of bit errors. The priority of the bits in a progressive video bitstream is, by design, well defined. Thus, efficient channel coding schemes can be designed. In this paper, we propose an unequal error protection scheme in the application layer for the video bitstream that works in concatenation with the channel codes in the physical layer of state-of-the-art cellular systems and demonstrate that progressive video streaming can be achieved over standard compliant second generation (2G) (and beyond) cellular systems without having to modify the physical layer of the systems. The channel protection scheme described would also work for a layered coder or for a coder where one could prioritize the data from the most to the least important.

Traditional wavelet packet (WP) optimization techniques neglect information about the structure of the lossy part of the compression scheme. Such information, however, can help guide the optimization procedure so as to result in efficient WP structures. We propose a wavelet packet algorithm with a constrained rate-distortion optimization which makes it suited to subsequent tree-structured coding such as with the set partitioning in hierarchical trees (SPMT) algorithm. The (octave-band) wavelet transform lends itself to simple and coherent tree-shaped spatial relations which can then be used to define zero-trees. Yet, input images have different frequency distributions and an adaptive transform such as WP is bound to be more efficient on an image-by-image basis. With WP algorithms, the coefficients in the WP domain can be rearranged to produce what resembles (or simulates) the normal wavelet transform structure. This stage is usually performed to simplify the coding stage. However, an unconstrained optimization can result in a transformed image with complicated or incoherent tree-shaped spatial relations. This work aims to show that the efficiency of embedded coders such as SPMT and Shapiro's Zerotrees strongly depends on WP structures with coherent spatial tree relationships.

A dense motion field provides a number of advantages over the traditional block-based motion models in interpreting the temporal correlation between video frames. However, until now, its application to motion-compensated video coding has been limited due to the inhibitive rate needed to encode the motion field. We present an efficient scheme for encoding a dense motion field which makes it possible to apply it in a forward motion-compensated video coding framework. A multi-scale motion field is first estimated to represent the motion at different scales. This full multi-scale motion field is then reduced adaptively by a tree pruning algorithm, resulting in a motion field that is represented at different scales from region to region. The pruned motion vector components are coded by a backward context-based arithmetic coding scheme. The motion vector encoding scheme is then incorporated into a complete video codec in which a much greater percentage of the overall bit-rate is allocated to the motion information (+30%). Simulations show that such a system provides substantial improvements over standard block-based coders.

The growth of new imaging technologies has created a need for techniques that can be used for copyright protection of digital images and video. One approach for copyright protection is to introduce an invisible signal, known as a digital watermark, into an image or video sequence. In this paper, we describe digital watermarking techniques, known as perceptually based watermarks, that are designed to exploit aspects of the the human visual system in order to provide a transparent (invisible), yet robust watermark. In the most general sense, any watermarking technique that attempts to incorporate an invisible mark into an image is perceptually based. However, in order to provide transparency and robustness to attack, two conflicting requirements from a signal processing perspective, more sophisticated use of perceptual information in the watermarking process is required. We describe watermarking techniques ranging from simple schemes which incorporate common-sense rules in using perceptual information in the watermarking process, to more elaborate schemes which adapt to local image characteristics based on more formal perceptual models. This review is not meant to be exhaustive; its aim is to provide the reader with an understanding of how the techniques have been evolving as the requirements and applications become better defined.

We study certain aspects of the problem of watermarking images, not for the classical example of ownership identification, but instead for embedding a unique identifier that can act as a "serial number" for the image in which it is embedded. In this context, two fundamental questions are: (a) what is the maximum number of different watermarks that can be distinguished reliably? and (b) what are good design techniques for watermarking methods to approximate that maximum? Under the assumption that attacks can be modeled as additive noise, we provide answers to both questions. To answer (a), we first show how the process of inserting a watermark is analogous to that of storing bits in certain devices, and then we compute the storage capacity of these devices. To answer (b), we present a specific design of a modulator to store information in those devices. Numerical simulations reveal that although strongly suboptimal in the number of bits they can effectively store in an image, simple and very low complexity modulator designs are able to pack enough bits in an image to be useful in practice.

The growth of networked multimedia systems has complicated copyright enforcement relative to digital images. One way to protect the copyright of digital images is to add an invisible structure to the image (known as a digital watermark) to identify the owner. In particular, it is important for Internet and image database applications that as much of the watermark as possible remain in the image after compression. Image adaptive watermarks are particularly resistant to removal by signal processing attack such as filtering or compression. Common image adaptive watermarks operate in the transform domain (DCT or wavelet); the same domains are also used for popular image compression techniques (JPEG, EZW). This paper investigates whether matching the watermarking domain to the compression transform domain will make the watermark more robust to compression.

The huge success of the Internet allows for the transmission, wide distribution, and access of electronic data in an effortless manner. Content providers are faced with the challenge of how to protect their electronic data. This problem has generated a flurry of research activity in the area of digital watermarking of electronic content for copyright protection. The challenge here is to introduce a digital watermark that does not alter the perceived quality of the electronic content, while being extremely robust to attack. For instance, in the case of image data, editing the picture or illegal tampering should not destroy or transform the watermark into another valid signature. Equally important, the watermark should not alter the perceived visual quality of the image. From a signal processing perspective, the two basic requirements for an effective watermarking scheme, robustness and transparency, conflict with each other. We propose two watermarking techniques for digital images that are based on utilizing visual models which have been developed in the context of image compression. Specifically, we propose watermarking schemes where visual models are used to determine image dependent upper bounds on watermark insertion. This allows us to provide the maximum strength transparent watermark which, in turn, is extremely robust to common image processing and editing such as JPEG compression, rescaling, and cropping. We propose perceptually based watermarking schemes in two frameworks: the block-based discrete cosine transform and multiresolution wavelet framework and discuss the merits of each one. Our schemes are shown to provide very good results both in terms of image transparency and robustness.

Content providers on the Internet are faced with the problem of how to secure electronic data. This problem has generated research activity in the area of digital watermarking of electronic content. The challenge is to introduce a digital watermark that is both transparent and highly robust to common signal processing and possible attacks. The two basic requirements for an effective watermarking scheme, robustness and transparency, conflict with each other. We propose a watermarking technique for digital images that is based on utilizing visual models which have been developed in the context of image compression. The visual models give us a direct way to determine the maximum amount of watermark signal that each portion of an image can tolerate without affecting the visual quality of the image. This allows us to provide the maximum strength watermark which in turn, is extremely robust to common image processing and editing such as JPEG compression, rescaling, and cropping. Our watermarking scheme is based on a DCT framework which allows for the possibility of directly watermarking the JPEG bitstream. Our scheme is shown to provide very good results both in terms of image transparency and robustness.

Face recognition has many applications ranging from security access to video indexing by content. We describe an automatic face recognition system which is VQ-based and examine the effects of feature selection, feature dimensionality and codebook size on recognition performance in the VQ framework. In particular, we examine DCT-based feature vectors in such a system. DCT-based feature vectors have the additional appeal that the recognition can be performed directly on the bitstream of compressed images which are DCT-based. The system described consists of three parts: a preprocessing step to segment the face, the feature selection process and the classification. Recognition rates for a database of 500 images shows promising results.

Color pictures are usually compressed in a luminance-chrominance coordinate space. We consider the problem of encoding the chrominance information for very low bit rate video coding systems aimed at bit rates in the range 8 to 40 kbps. The challenge is that the chrominance components typically get less than 10 to 20% of the total very low bit rate allocated for the video data. We found that it is sufficient to encode the chrominance information at 1/8 of the luminance resolution in both the horizontal and vertical directions. While, for many of the previous coding methods, the compression is performed independently for the luminance and chrominance coordinates, we propose a coding scheme which exploits the coded luminance data in coding and retrieving the chrominance components. The proposed video coder is an improved extension of an existing luminance-only coder so that color motion video can be coded at very low bit rates under fixed frame and bit rate constraints. It is based on a hybrid waveform coding technique with an implicit model-based component. Very good results were obtained for head-and-shoulders sequences even with chroma rates of less than 7% of the total very low bit rate. In addition, subjective tests indicate that the coded chrominance information improves the visual perception of noisy image features.

We describe and show the results of video coding based on a three-dimensional (3-D) spatio-temporal subband decomposition. The results include a 1-Mbps coder based on a new adaptive differential pulse code modulation scheme (ADPCM) and adaptive bit allocation. This rate is useful for video storage on CD-ROM. Coding results are also shown for a 384-kbps rate that are based on ADPCM for the lowest frequency band and a new form of vector quantization (geometric vector quantization (GVQ)) for the data in the higher frequency bands. GVQ takes advantage of the inherent structure and sparseness of the data in the higher bands. Results are also shown for a 128-kbps coder that is based on an unbalanced tree-structured vector quantizer (UTSVQ) for the lowest frequency band and GVQ for the higher frequency bands. The results are competitive with traditional video coding techniques and provide the motivation for investigating the 3-D subband framework for different coding schemes and various applications.

Video coding using a three-dimensional subband decomposition is described and results are shown for 64 kbps. The three-dimensional subband structure allows for adaptive bit allocation between temporal and spatial information. Conditional replenishment is implemented in the low-temporal frequency bands to take advantage of temporal correlations. The high-frequency information is efficiently encoded using geometric vector quantization (GVQ) which takes advantage of the sparse structured data typically found in the high frequency bands. Coding results at 64 kbps on head-and-shoulder sequences typical of video teleconferencing applications are compared with results obtained with the CCITT Recommendation H.261 coder.>

The full motion video described is based on the efficient coding of three dimensional subband data. The resulting bit rates of this coder include 384 kb/s results for image sequences of medium complexity and 128 kb/s results for simpler image sequences. The image data are decomposed into spatiotemporal frequency bands and different quantization schemes are applied to each frequency band based on the characteristics of the data in that band. The lowest spatiotemporal frequency band is coded using an unbalanced tree-structured vector quantization technique where the temporal and spatial correlations are exploited in a lossless coding technique referred to as interblock noiseless coding.>

A full-motion video coder based on a threedimensional subband framework is developed. The original image data is decomposed into different spatial-temporal frequency bands and based on the characteristics of the data in each band, different quantization strategies are applied to each band. The lowest spatial-temporal frequency band is coded using a three dimensional switched predictor and optimum scalar quantizer. The nondominant subbands which contain perceptually significant data are coded using a vector quantization approach in which the sparse, highly structured nature of the data in the higher-frequency subbands is exploited in the design of the codebook.>