The paper looks into multi-channel/multimodal biometric systems that are adaptive to environmental variations. We introduce a general formulation that addresses the environmental robustness of multi-channel fusion in biometric systems. Based on the formulation, two audio-visual biometric systems are developed. The first relies on confidence measures derived from environmental conditions to weight the contributions of the biometric channels dynamically; whereas the second considers the multiple channels jointly to adjust the fusion parameters optimally according to the current environmental conditions. Experimental evaluations with varying testing conditions show that both systems achieve lower recognition error rate compared with a baseline non-environment-adaptive audio-visual system. It is further shown that incorporating joint optimization of multichannel fusion parameters to cater to environmental changes, as in the second system, consistently leads to improved recognition accuracy over other systems, and, at the same time, it is guaranteed to perform no worse than any of the individual biometric channels under all environmental conditions.

We propose a new paradigm for implementations of audio array processing algorithms on a network of distributed general-purpose computers. In contrast to currently existing DSP processor-based solutions, our approach offers new possibilities for advanced array signal processing by enabling the usage of general-purpose computing platforms with their superior computational and storage resources. We demonstrate that synchronization of sensors is essential for acoustic blind source separation (BSS) algorithms, and we propose a synchronization scheme that enables BSS on distributed, wirelessly networked computers and can easily be implemented on existing hardware.

Media applications have been driving microprocessor development for more than a decade. Future multimedia applications will have much higher computational requirements. Indeed, tomorrow's PC experiences will be even richer in audio-visual effects, easier to use, and, more importantly, computing will merge with communications. Such application characteristics will differ from current application characteristics and thus, a new detailed analysis of such kernels is required. This paper proposes a selection of key kernels we believe will be prominent across applications running on future processors. An extensive study of these applications on current general-purpose processor architectures makes it possible to determine future media and communication characteristics and to identify bottlenecks that are not well addressed in current designs.

This paper presents the study and results of running several core multimedia applications on a simultaneous multithreading (SMT) architecture, including some detailed analysis ranging from memory-bounded kernels to computational-bounded functions. A performance metric to evaluate effective SMT performance gain is introduced, and compared to similar metrics on symmetric multiprocessor (SMP) systems. In addition, we analyze and compare SMT versus SMP systems, and highlight the advantages in the studied applications. The results indicate that sharing the cache in SMT processors can provide better cache locality and thus better performance although sharing the cache can introduce cache conflicts and reduce the actual cache size available for each logical processor. We also propose "mutual prefetching" -a technique to schedule threads so that they prefetch data for each other in order to reduce cache miss penalty.

We address the problem of real-time video streaming over wireless LANs for both unicast and multicast transmission. The wireless channel is modeled as a packet-erasure channel at the IP level. For the unicast scenario, we describe a novel hybrid Automatic Repeat reQuest (ARQ) algorithm that efficiently combines forward error control (FEC) coding with the ARQ protocol. For the multiple-users scenario, we formulate the problem of real-time video multicast as an optimization of a maximum regret cost function across the multicast user space. The proposed solution efficiently combines progressive source coding with FEC coding. We present a theoretical analysis of the unicast and multicast cases, as well as experimental results that demonstrate the performance advantages of the proposed algorithms over existing methods.

There has been increasing interest in 3D geometry compression. One reason is that the insufficient bandwidth between the graphics pipeline and the main memory is not able to handle the huge amount of 3D graphics data transferred between these two components. A variation of open-loop linear predictive quantization (OLPQ), a widely used but not well-studied coding technique, was proposed to become the geometry coding technique of MPEG4 3D model coding (3DMC). We compare the performance of OLPQ with the well-understood differential pulse code modulation (DPCM). Our results show that DPCM does not outperform OLPQ in the coding of vertex coordinates of 3D objects in the rate-distortion sense. Such results together with the lower complexity of OLPQ makes OLPQ a better geometry coding technique than DPCM.

We present a new approach for faster rendering of large synthetic environments using video-based representations. We decompose the large environment into cells and pre-compute video based impostors using MPEG compression to represent sets of objects that are far from each cell. At runtime, we decode the MPEG streams and use rendering algorithms that provide nearly constant-time random access to any frame. The resulting system has been implemented and used for an interactive walkthrough of a model of a house with 260,000 polygons and realistic lighting and textures. It is able to render this model at 16 frames per second (an eightfold improvement over simpler algorithms) on average on a Pentium II PC with an off-the-shelf graphics card.

There have been many works on watermarking of 2D images and video. However, watermarking of three-dimensional (3D) objects has received little attentions. We introduce and investigate the watermarking of three-dimensional (3D) objects as compared to 2D images, study the watermarking of objects represented by polygonal models, and propose a new fragile watermarking technique that enables the verification of 3D polygonal models for detecting unauthorized alterations. Our proposed technique consists of a watermark encoding process which embeds a sequence of data bits by perturbing coordinates of the vertices in a 3D model, and a watermark decoding process which extracts bit information from 3D geometry, using secure verification keys. Fast 2D visualization of unauthorized modification to 3D objects is also proposed.

Digital watermarks have been proposed as a means for copyright protection of multimedia data. We address the capability of invisible watermarking schemes to resolve copyright ownership. We show that, in certain applications, rightful ownership cannot be resolved by current watermarking schemes alone. Specifically, we attack existing techniques by providing counterfeit watermarking schemes that can be performed on a watermarked image to allow multiple claims of rightful ownership. In the absence of standardization and specific requirements imposed on watermarking procedures, anyone can claim ownership of any watermarked image. In order to protect against the counterfeiting techniques that we develop, we examine the properties necessary for resolving ownership via invisible watermarking. We introduce and study invertibility and quasi-invertibility of invisible watermarking techniques. We propose noninvertible watermarking schemes, and subsequently give examples of techniques that we believe to be nonquasi-invertible and hence invulnerable against more sophisticated attacks proposed in the paper. The attacks and results presented in the paper, and the remedies proposed, further imply that we have to carefully reevaluate the current approaches and techniques in invisible watermarking of digital images based on application domains, and rethink the promises, applications and implications of such digital means of copyright protection.

We shall show that non-invertibility is a necessary but not sufficient condition in resolving ownership disputes. We then define quasi-invertible watermarking schemes, and, present analysis that links invertibility and quasi-invertibility to some classes of watermarking techniques with different properties (which may or may not require original versions in watermark decoding), as well as to the different classes of attacks we have developed.

We have entered an era where inexpensive and readily-available equipment can produce perfect copies of digital multimedia materials, such as CD-quality audio, publication-quality images, or digital video. In this environment, it has become easier for malicious parties to make salable copies of copyrighted content without compensation to the content owner. Many media content owners are concerned about the potential loss of revenue from multimedia piracy, especially when the content will be exposed to the Internet. Digital watermarking is seen by many as a potential solution to this problem. Many different watermarking schemes have been proposed. Often, however there is little discussion of how effective a proposed watermarking technique may be at solving a particular problem. We describe a number of proposed image-watermarking application scenarios and form a small number of watermark-application categories. Then, with these applications in mind, we discuss the desired technical properties of watermarks for each category. Finally we discuss some watermarking techniques developed by the authors, in light of the desired properties.

We are witnessing today increasing use and popularity of digital video, from CD ROMs, Internet to digital TV broadcast. However as with analog video, these applications treat video as a sequence of moving images and the primary means of playback is sequential in nature, from the first to last frame. We describe new applications of creating and using digital video and discuss new paradigms of analysis and synthesis that spearheaded these applications. We highlight research issues and directions in and beyond image processing on the analysis and synthesis of digital video for a new generation of applications.

We propose a new method for invisibly watermarking high-quality color and gray-scale images. This method is intended for use in image verification applications, where one is interested in knowing whether the content of an image has been altered since some earlier time, perhaps because of the act of a malicious party. It consists of both a watermark stamping process which embeds a watermark in a source image, and a watermark extraction process which extracts a watermark from a stamped image. The extracted watermark can be used to determine whether the image has been altered. The processing used in the stamping and extraction processes is presented. We also discuss some advantages of this technique over other invisible watermarking techniques for the verification application; these include a high degree of invisibility, color preservation, ease of decoding, and a high degree of protection against retention of the watermark after unauthorized alterations.

In this paper, we address the requirements, techniques and applications of digital watermarking for high-quality images. We present several classes of digital watermarking techniques defined by their appearance and application domains, specify the requirements, and summarize the current research efforts in the Image Library Applications group of the IBM Thomas J. Watson Research Center on digital watermarking of high-quality images. We hope that by addressing these issues, we can both provide a foundation for the next generation of digital watermarking techniques, and a vision for their future applications.

Digital video archives are likely to be accessible on distributed networks which means that the data are subject to network congestion and bandwidth constraints. To enable new applications and services of digital video, it is not only important to develop tools to analyze and browse video, view query results, and formulate better searches, but also to deliver the essence of the material in compact forms. Video visualization describes the joint process of analyzing video and the subsequent derivation of representative visual presentation of the essence of the content. We propose techniques to analyze video and build a compact pictorial summary for visual presentation. A video sequence is thus condensed into a few images-each summarizing the dramatic incident taking place in a meaningful segment of the video. In particular, we present techniques to differentiate the dominance of the content in subdivisions of the segment based on analysis results, select a graphic layout pattern according to the relative dominances, and create a set of video posters, each of which is a compact, visually pleasant, and intuitive representation of the story content. The collection of video posters arranged in temporal order then forms a pictorial summary of the sequence to tell the underlying story. The techniques and compact presentations proposed offer valuable tools for new applications and services of digital video including video browsing, query, search, and retrieval in the digital libraries and over the Internet.

Many video programs have story structures that can be recognized through the clustering of video contents based on low-level visual primitives, and the analysis of high level structures imposed by temporal arrangement of composing elements. In this paper time-constrained clustering of video shots is proposed to collapse visually similar and temporally local shots into a compact structure. We show that the proposed clustering formulations, when incorporated into the scene transition graph framework, allows the automatic segmentation of scenes and story units that cannot be achieved by existing shot boundary detection schemes. The proposed method is able to decompose video into meaningful hierarchies and provide compact representations that reflect the flow of story, thus offering efficient browsing and organization of video.

Browsing, search and retrieval in digital video libraries depend on the ability of the system to match, classify and group video shots by their visual contents. However, similarity of shots cannot always be settled by using only one key frame per shot as commonly practiced. In this paper we propose a scheme to match video shots and to cluster them by taking into account the temporal variations within individual shots. A much reduced representation for a video shot is used. These images still capture the dynamics of visual contents for matching and clustering process. Experimental results are reported.

This paper presents a 3D image registration approach to 3D CT angiography (CTA). We extend the interest operator from 2D to 3D in order to extract 3D feature points and develop a 3D image flow algorithm to find matches of feature points between the mask and the contrast data sets. A robust 3D motion estimation algorithm is also developed which handles outliers up to 50% of all computed image flow vectors. The subtracted volume after motion correction is rendered on the display. The proposed algorithm has been successfully applied to a clinical CTA data set and the result is very promising.>

Proposes a low-complexity fixed-rate vector quantizer for DCT transformed images based on the packing of scalar quantized DCT coefficients into pyramid-bounded integer lattices. A closed form expression for the size of pyramid-bounded integer lattices is derived and a method of bit-rate allocation to the various coefficients is described. The proposed scheme is simple and has performed well for various bit rates.>