Advances in graphics and machine learning have led to the general availability of easy-to-use tools for modifying and synthesizing media. The proliferation of these tools threatens to cast doubt on the veracity of all media. One approach to thwarting the flow of fake media is to detect modified or synthesized media through machine learning methods. While detection may help in the short term, we believe that it is destined to fail as the quality of fake media generation continues to improve. Soon, neither humans nor algorithms will be able to reliably distinguish fake versus real content. Thus, pipelines for assuring the source and integrity of media will be required---and increasingly relied upon. We present AMP, a system that ensures the authentication of media via certifying provenance. AMP creates one or more publisher-signed manifests for a media instance uploaded by a content provider. These manifests are stored in a database allowing fast lookup from applications such as browsers. For reference, the manifests are also registered and signed by a permissioned ledger, implemented using the Confidential Consortium Framework (CCF). CCF employs both software and hardware techniques to ensure the integrity and transparency of all registered manifests. AMP, through its use of CCF, enables a consortium of media providers to govern the service while making all its operations auditable. The authenticity of the media can be communicated to the user via visual elements in the browser, indicating that an AMP manifest has been successfully located and verified.

The popularization of video capture devices has created strong storage demand for encoded videos. Approximate storage can ease this demand by enabling denser storage at the expense of occasional errors. Unfortunately, even minor storage errors, such as bit flips, can result in major visual damage in encoded videos. Similarly, video encryption, widely employed for privacy and digital rights management, may create long dependencies between bits that show little or no tolerance to storage errors.

This is a somewhat unusual paper. It combines some history of the author's personal experiences with Prof. Sanjit Mitra with a couple of examples of the author's work, in which the author feels that Prof. Mitra's work and approaches towards research and impact have had a significant influence.

This paper proposes tailoring image encoding for an approximate storage substrate. We demonstrate that indiscriminately storing encoded images in approximate memory generates unacceptable and uncontrollable quality degradation. The key finding is that errors in the encoded bit streams have non-uniform impact on the decoded image quality. We develop a methodology to determine the relative importance of encoded bits and store them in an approximate storage substrate. The storage cells are optimized to reduce error rate via biasing and are tuned to meet the desired reliability requirement via selective error correction. In a case study with the progressive transform codec (PTC), a precursor to JPEG XR, the proposed approximate image storage system exhibits a 2.7x increase in density of pixels per silicon volume under bounded error rates, and this achievement is additive to the storage savings of PTC compression.

Summary form only given. High-resolution color video (in 4:4:4 format) is becoming increasingly important for screen content and graphics. We have developed a frame packing arrangement (FPA) scheme that enables transmission of 4:4:4 luma-chroma (YUV or YCoCg) content through conventional subsampled chroma (4:2:0) codecs as an approach for low-cost support of 4:4:4 content. We focus on the "band separation" method initially discussed in [1] and evaluate new scenarios with the 4:4:4 and 4:2:0 versions at similar compression levels. We use the HEVC HM12.0 reference software for evaluating the FPA scheme. We vary the luma QP for the main 4:2:0 representation from 10 to 34, keeping the QP for the main frame's chroma and auxiliary frame's luma and chroma (containing three-fourths of the 4:4:4 chroma data) at the same value as the luma QP. We then compare the final 4:4:4 chroma rate-distortion performance of band separation, direct frame packing, and native HM 4:4:4 encoding. We also show preliminary results for lifting-based band separation (with clipping), which eliminates the inherent rounding errors of other band separation filters and provides better rate-distortion performance. Fig. 1 provides an example for a typical case of the new results showing that both the "band separation" and "lifting-based band separation" approaches have much better performance than previously reported, and are significantly better than the earlier-developed "direct" frame packing approach. Although the rate-distortion performance is inferior to that of native HM 4:4:4 coding, the FPA scheme enables the use of existing 4:2:0 decoding hardware, thus allowing for considerable battery power savings in mobile devices. A more complete discussion and evaluation is available in [2].

We present a method to convey high-resolution color (4:4:4) video content through a video coding system designed for chroma-sub sampled (4:2:0) operation. The method operates by packing the samples of a 4:4:4 frame into two frames that are then encoded as if they were ordinary 4:2:0 content. After being received and decoded, the packing process is reversed to recover a 4:4:4 video frame. As 4:2:0 is the most widely supported digital color format, the described scheme provides an effective way of transporting 4:4:4 content through existing mass-market encoders and decoders, for applications such as coding of screen content. The described packing arrangement is designed such that the spatial correspondence and motion vector displacement relationships between the nominally-luma and nominally-chroma components are preserved. The use of this scheme can be indicated by a metadata tag such as the frame packing arrangement supplemental enhancement information (SEI) message defined in the HEVC and AVC (Rec. ITU-T H.264 | ISO/IEC 14496-10) video coding standards. In this context the scheme would operate in a similar manner as is commonly used for packing the two views of stereoscopic 3D video for compatible encoding. The technique can also be extended to transport 4:2:2 video through 4:2:0 systems or 4:4:4 video through 4:2:2 systems.

We present a low-complexity integer-reversible spectral-spatial transform that allows for efficient loss less and lossy compression of color-filter-array images (also referred to as camera-raw images). The main advantage of this new transform is that it maps the pixel array values into a format that can be directly compressed in a loss less, lossy, or progressive-to-loss less manner by an existing typical image coder such as JPEG 2000 or JPEG XR. Thus, no special codec design is needed for compressing the camera-raw data. Another advantage is that the new transform allows for mild compression of camera-raw data in a near-loss less format, allowing for very high quality offline post-processing, but with camera-raw files that can be half the size of those of existing camera-raw formats.

We propose a new transform speech codec that jointly encodes a wideband waveform and its corresponding wideband and narrowband speech recognition features. For distributed speech recognition, wideband features are compressed and transmitted as side information. The waveform is then encoded in a manner that exploits the information already captured by the speech features. Narrowband speech acoustic features can be synthesized at the server by applying a transformation to the decoded wideband features. An evaluation conducted on an in-car speech recognition task show that at 16 kbps our new system typically shows essentially no impact in word error rate compared to uncompressed audio, whereas the standard transform codec produces up to a 20% increase in word error rate. In addition, good quality speech is obtained for playback and transcription, with PESQ scores ranging from 3.2 to 3.4.

We explore several local and global strategies for adaptive scan ordering of transform coefficients in JPEG XR/HD Photo. This codec applies a global adaptive scan-order heuristic with respect to an effective localized predictor. The global ordering heuristic, although simple, performs as well as localized techniques that are computationally significantly more complex. We conclude that effective localized prediction not only minimizes but also essentially randomizes coefficient residuals, so that a global statistic is sufficient to deliver near-optimal compression performance.

This paper describes a method for designing oversampled DFT filter banks (FB) optimized for subband acoustic echo cancellation (AEC). For this application, the design requirements typically are good echo cancellation quality, low delay, small reconstruction error, and low computation complexity. Our method explicitly includes a model for echo return loss enhancement (ERLE) as part of the optimization criteria. Convergence of the high-dimensional, nonlinear optimization problem is facilitated by decorrelating the prototype filter impulse response via a discreet cosine transform (DCT), discarding many insignificant coefficients and thus reducing the dimensionality of the search. The experimental results demonstrate the effectiveness of our design and the effects on the performance of AEC, with ELRE improvements on the order of 3 dB or better. The method is flexible and could also be extended to other application domains.

We propose a system for audio coding using the modulated complex lapped transform (MCLT). In general, it is difficult to encode signals using overcomplete representations without avoiding a penalty in rate-distortion performance. We show that the penalty can be significantly reduced for MCLT-based representations, without the need for iterative methods of sparsity reduction. We achieve that via a magnitude-phase polar quantization and the use of magnitude and phase prediction. Compared to systems based on quantization of orthogonal representations such as the modulated lapped transform (MLT), the new system allows for reduced warbling artifacts and more precise computation of frequency-domain auditory masking functions.

Linear adaptive filters are often used for acoustic echo cancellation (AEC) but sometimes fail to perform well in notebook computers and inexpensive telephony devices. Low-quality speakers and poorly-designed enclosures that produce vibrations often generate harmonic distortion, and this nonlinear effect degrades the performance of linear AEC algorithms considerably. In this work, we present a new AEC architecture that consists of a linear, subband adaptive AEC filter followed a nonlinear residual echo suppression (RES) stage specifically designed to address harmonic distortion. In addition to suppressing the residual echo in the primary subband, the proposed model also suppresses the residual echo in a window of bands surrounding the higher order harmonics. Results show considerable improvement over other proposed algorithms, and the new algorithm has much lower implementation costs compared to nonlinear AEC models based on Volterra filters and a previously proposed, nonlinear residual echo suppression algorithm.

We present a simple lossless audio codec, composed of an integer-reversible modulated lapped transform (MLT) followed by a backward-adaptive run-length/Golomb-Rice (RLGR) encoder. Its compression performance matches those of state-of-the-art predictive codecs, and it has the advantage that its compressed bitstream contains frequency-domain data that can be used for applications such as search, identification, and visualization. Its compression gain can be improved through a novel data model based on cross-block smoothed spectral magnitude estimates. Its bitstream can be transcoded into a lossy format, for transfers to portable players, at about twice the speed of other codecs. The codec also supports a near-lossless mode, which allows for an extra factor of two in compression without noticeable distortions

It is usual in practice that recorded sounds are contaminated by stationary tones coming from power wiring (50/60 Hz or 400 Hz and their harmonics), frame or line frequencies from a nearby TV or monitor, computer fans, hard disk drives, etc. They are mostly stationary, but their removal using a stationary noise suppressor results in notch filtering, removing the speech content at those frequencies, because the SNRs are usually low. In this paper we propose fast, real-time algorithm for removing constant tones while keeping intact the speech content. We build and adaptively update a model of the constant tones, extrapolating it for subtraction from the next frame. In our evaluations, the proposed algorithm reduces the amplitudes of such stationary tones up to 15 dB, without introducing artifacts such as nonlinear distortions or musical noises. This algorithm is applicable as pre-processor before a classic gain-based stationary noise suppressor.

We present a simple and efficient entropy coder that combines run-length and Golomb-Rice encoders. The encoder automatically switches between the two modes according to simple rules that adjust the encoding parameters based on the previous output codeword, and the decoder tracks such changes. This adaptive run-length/Golomb-Rice (RLGR) coder has a fast learning rate, making it suitable for many practical applications, which usually involve encoding small source blocks. We study the encoding of generalized Gaussian (GG) sources after quantization with uniform scalar quantizers with deadzone, which are good source models in multimedia data compression, for example. We show that, for a wide range of source parameters, the RLGR encoder has a performance close to that of the optimal Golomb-Rice and exp-Golomb coders designed with knowledge of the source statistics, and in some cases the RLGR coder improves coding efficiency by 20% or more.

We propose a simple method for the design of lapped transforms with nonuniform frequency resolution and good time localization. The method is a generalization of an approach previously proposed by Princen, where the nonuniform filter bank is obtained by joining uniform cosine-modulated filter banks (CMFBs) using a transition filter. We use several transition filters to obtain a near perfect-reconstruction (PR) nonuniform lapped transform with significantly reduced overall distortion. The main advantage of the proposed method is in reducing the length of the transition filters, which leads to a reduction in processing delay that can be useful for applications such as real-time audio coding.

Audio fingerprinting is a powerful tool for identifying file-based or streaming audio, using a database of fingerprints. The paper presents two new applications of audio fingerprinting: duplicate detection, whose goal is to identify duplicate audio clips in a set, even if they differ in compression quality or duration, and thumbnail generation, which aims to provide a representative short clip of a music track. Neither application requires an external database of fingerprints. Thanks to the robustness of the fingerprinting engine, both applications perform well; the duplicate detector has a false positive rate that is conservatively bounded above by 1% on a very large data set, and the thumbnail generator significantly outperforms using a fixed window.

This paper presents a generic beamformer design algorithm for arbitrary microphone array geometry. It makes efficient use of noise models for ambient and instrumental and microphone directivity patterns. By using a new definition of the target criterion and replacing a multi-dimensional optimization with a much simpler one-dimensional search, we can compute near-optimal solutions in reasonable time. The designed beams achieve noise suppression levels between 10 and 15 dB, for microphone arrays with four to eight elements, and linear and circular geometries. The fast beamformer real-time processing engine consumes less than 2% of the CPU power of a modern personal computer, for a four-microphone array.

This paper describes an efficient run-length encoding of binary sources with unknown statistics. Binary entropy coders are used in many multimedia codec standards, which uses adaptive Golomb-Rice coders. Using a maximum-likelihood approach, an excess rate for the Golomb-like coder when compared to an adaptive Rice coder is up to 4.2% for binary sources with unknown statistics with respect to the source entropy.

Many bitmap documents are composed by the superposition of layers with pictures and text. These documents do not compress well using image compression algorithms such as JPEG-2000, because text introduces sharp edges on top of the smooth surfaces typically found in natural images. Similarly, compression algorithms for text facsimiles, such as JBIG2, are not suited for color or gray level images. In this paper the SLIm system for separating text and line drawing from background images, in order to compress both more effectively is proposed. This approach differ from previous ones such as DjVu, Tiff-FX, and MRC, by being extremely simple and fast, while yielding close to state-of-the-art compression performance. Results show that the SLIm compression performance is attractive for many applications.

This paper introduces a new interpolation technique for demosaicing of color images produced by single-CCD digital cameras. We show that the proposed simple linear filter can lead to an improvement in PSNR of over 5.5 dB when compared to bilinear demosaicing, and about 0.7 dB improvement in R and B interpolation when compared to a recently introduced linear interpolator. The proposed filter also outperforms most nonlinear demosaicing algorithms, without the artifacts due to nonlinear processing, and a much reduced computational complexity.

Activity detection on binary images can be a useful part of image processing for detecting noise, texture, printed text, or dithering. We present an image activity detector based on computing a density of selected connected components (CCs). Connectedness is a useful property because it is present in individual printed letters, lines, and edges. In contrast, salt-and-pepper noise and dithering are typically composed of a large number of disconnected patterns. By filtering the CCs based on size, we can measure different kinds of activities, and segment or filter the image accordingly. The activity detector is extremely efficient and can be run in a fraction of the time it takes to compute a run-length encoding version of the image. As an example, we built a noise removal filter based on the density of CCs, which is both faster and better than a conventional median filter.

This paper introduces a new architecture for implementing subband acoustic echo cancellation (AEC) with arbitrary playback sampling rate. Typically, in AEC algorithms for audio or videoconferencing, the sampling rates for the signals played through the speakers and captured from the microphones are identical. For speech recognition while playing CD-quality music and Internet gaming with voice chat, the playback sampling rate is usually higher than the capture rate. A direct solution is to apply a sampling rate converter to the playback signal before feeding it to the AEC, but that is complicated if many sampling frequencies must be supported. We propose a more efficient solution for subband AEC: we perform the sampling rate conversion as a frequency-domain interpolation that matches the transform lengths of the playback and capture signals. Results show that the new AEC architecture has a small computational cost and only a minimal reduction in echo attenuation.

Images typically contain smooth regions, which are easily compressed by linear transforms, and high activity regions (edges, textures), which are harder to compress. To compress the first kind, we use a "zero" encoder that has infinite context, very low capacity, and which adapts very quickly to the content. For the second, we use an "interpolation" encoder, based on neural networks, which has high capacity, a finite-size context, and is trained off-line. The two encoders can be used separately or in combination. The zero-encoder surpasses JPEG2000 by 3.5% in overall compression, even though it is less efficient in high activity regions. Thanks to off-line training, the interpolation-encoder predicts high activity regions well, so it also matches the performance of JPEG2000, even though it does not use an arithmetic encoder and is less efficient in low activity regions. In both cases it is surprising that we match the state-of-the-art in image compression without using adaptive arithmetic encoding.

A new algorithm for fast computation of the modulated complex lapped transform (MCLT) is presented. For a length-M MCLT, the algorithm is based on computing a length-2M fast Fourier transform plus M butterfly-like stages, without data shuffling. That reduces the number of operations and memory accesses needed to compute the MCLT when compared to previous algorithms. Compared to the original type-4 discrete-cosine-transform-based algorithm, an implementation of the new algorithm leads to 25% faster execution.

This paper presents an overview of the transform and quantization designs in H.264. Unlike the popular 8/spl times/8 discrete cosine transform used in previous standards, the 4/spl times/4 transforms in H.264 can be computed exactly in integer arithmetic, thus avoiding inverse transform mismatch problems. The new transforms can also be computed without multiplications, just additions and shifts, in 16-bit arithmetic, thus minimizing computational complexity, especially for low-end processors. By using short tables, the new quantization formulas use multiplications but avoid divisions.

On-line adaptation to nonstationary distributions is essential to good performance in image coding. Fixed-size contexts (with adaptive tables) are also widely used, in conjunction with arithmetic encoders, in state-of-the-art codecs. In contrast, we propose a simple two-dimensional filter that directly outputs the probability distribution function (PDF) estimate necessary to drive an adaptive arithmetic encoder. The filter is isotropic, in the sense that the impact of a previously encoded bit depends only on its value and distance to the bit to be coded. Surprisingly, this simple filter yields results comparable to or better than JPEG2000. It also brings an interesting distinction between on-line and off-line learning, and their relative importance in compression.

Spread-spectrum (SS) watermarking has been one of the oldest methodologies for hiding data in multimedia signals. Recently, it has been demonstrated that simple block repetition codes provide strong robustness of such watermarks to noise addition and limited arbitrary geometric transformations. In order to further investigate the security of such watermarks, we explore in this paper their robustness with respect to watermark estimation attacks. In such attacks, the adversary has knowledge of the watermark algorithm, except the secret keys. We present a modification of the traditional SS watermark detector that forces the adversary to increase the amount of noise to be proportional to the signal amplitude, in order to remove an SS watermark.

This paper introduces a new watermarking modulation technique, which we call improved spread spectrum (ISS). Unlike in traditional spread spectrum (SS), in ISS the carrier signal does not act as a noise source; leading significant performance gains. In typical examples, the gain of ISS over SS is 20 dB or more in signal-to-noise ratio or 20 orders of magnitude or more in error probability. The proposed method achieves roughly the same noise robustness gain as quantization index modulation (QIM). Nevertheless, while QIM is quite sensitive to amplitude variations, ISS is as robust in practice as traditional SS.

This paper presents an overview of the latest transform and quantization designs for H.26L. Unlike the popular discrete cosine transform (DCT) used in previous standards, the transforms in H.26L can be computed exactly in integer arithmetic, thus avoiding inverse transform mismatch problems. The new transforms can also be computed without multiplications, just additions and shifts, in 16-bit arithmetic, thus minimizing computational complexity, especially for low-end processors. By using short tables, the new quantization formulas use multiplications but avoid divisions.

We present a new dual watermarking and fingerprinting system, where initially all copies of a protected object are identically watermarked using a secret key, but individual detection keys are distinct. By knowing a detection key, an adversary cannot recreate the original content from the watermarked content. However, knowledge of any one detection key is sufficient for modifying the object so that a detector using that key would fail to detect the marks. Detectors using other detection keys would not be fooled, and such a modified object necessarily contains enough information about the broken detector key - the fingerprint. Our dual system limits the scope of possible attacks, when compared to classic fingerprinting systems. Under optimal attacks, the size of the collusion necessary to remove the marks without leaving a detectable fingerprint is superlinear in object size, whereas classic fingerprinting has a lower bound on collusion resistance that is approximately fourth root in object size. By using our scheme one can achieve collusion resistance of up to 900,000 users for a two hour high-definition video.

We present a new bi-level image compression coder that does not use arithmetic encoding, but whose performance is close to that of state-of-the-art coders such as JBIG, JBIG-2, and JB2. The proposed bi-level coder (BLC) uses two simple adaptation rules: the first to compute context-dependent probability estimates that control a pixel prediction module, and the second to adjust a run-length parameter in a run-length-Rice (RLR) coder. That's contrary to the usual approach where the context-dependent probability estimate controls both pixel prediction and adaptive entropy coding. Due to its simplicity, in many applications BLC may be a better choice than JBIG or JBIG-2.

This paper presents an algorithm for the compression of images that have "don't care" regions within them. Such regions are specified by an additional binary image, the mask, which we assume is available at both the coder and the encoder (it is transmitted separately and without loss). Unlike previous approaches, which typically fill in the masked pixels with values geared toward good compression of the image, we use a special wavelet transform which is a function of the mask, but which can be computed efficiently. We compare our method with the previous approaches by measuring the compression for a given PSNR on the valid pixels. Our algorithm is both faster and yields compression improvements of up to 18% over the current state-of-the-art coders.

Introduces an optimization criterion for the design of microphone arrays, and derives an optimum filter based on this criterion. The algorithm computes two separate correlation matrices for the signal: one for when only background noise is present, and one for when both noise and signal are present. A filter is then computed based on these matrices, optimizing the proposed weighted mean-square error criterion. A block-recursive version of the algorithm is presented, using LMS-like adaptation of the multichannel filters, with a computational complexity under 40 MIPS for a typical application with four microphones. Simulation results with typical office noise show improvements of up to 20 dB in signal-to-noise ratio, even in low-noise environments.

This paper presents an efficient algorithm for high-quality speech capture in applications such as hands-free teleconferencing or voice recording by personal computers. We process the microphone signals by a subband adaptive filtering structure using a modulated complex lapped transform (MCLT), in which the subband filters are adapted to maximize the kurtosis of the linear prediction (LP) residual of the reconstructed speech. In this way, we attain good solutions to the problem of blind speech dereverberation. Experimental results with actual data, as well as with artificially difficult reverberant situations, show very good performance, both in terms of a significant reduction of the perceived reverberation, as well as improvement in spectral fidelity.

We present several mechanisms that enable effective spread-spectrum audio watermarking systems: prevention against detection desynchronization, cepstrum filtering, and chess watermarks. We have incorporated these techniques into a system capable of reliably detecting a watermark in an audio clip that has been modified using a composition of attacks that degrade the original audio characteristics well beyond the limit of acceptable quality. Such attacks include: fluctuating scaling in the time and frequency domain, compression, addition and multiplication of noise, resampling, requantization, normalization, filtering, and random cutting and pasting of signal samples.

This paper explores blind deconvolution of reverberated speech signals in microphone array applications. Two regularization approaches are proposed based on available a priori knowledge. The regularized least-squares (LS) approach uses the speech signal characteristics and the lowpass nature of the reverberation channel; and the regularized cross correlation (CR) approach requires more precise knowledge of reverberation which can be obtained through training. The two methods are robust to the presence of noise.

Watermarking has been adopted as a technology of choice for many applications related to e-commerce of audio content. We present a brief summary of a set of spread-spectrum watermarking techniques for effective covert communication over an audio signal carrier. Watermark robustness is enabled using redundant spread-spectrum for prevention against de-synchronization attacks. We improve watermark inaudibility by detecting and not watermarking blocks of audio where a spread spectrum sequence, if added to the frequency spectrum, would be audible. Finally, we overview the security limitations of our technology with respect to parameter selection and position it with respect to three main applications of watermarking: (a) content screening, (b) tracing unlicensed content distribution, and (c) robust metadata.

A nonuniform modulated complex lapped transform (NMCLT) is introduced in this paper as a two-stage extension of the modulated complex lapped transform (MCLT). The NMCLT is a new nonuniform oversampled filter bank with a better combination of time- and frequency-domain localization than previous designs, Adaptive nonuniform subband decompositions can be easily generated by varying the number of coefficients brought to the second stage on a frame-by-frame basis. The NMCLT is ideally suited for processing wideband signals such as audio, with reduced time-spreading artifacts.

We introduce a new image compression algorithm that allows progressive image reconstruction - both in resolution and in fidelity, with a fully embedded bit-stream. The algorithm is based on bit-plane entropy coding of reordered transform coefficients, similar to the progressive wavelet codec (PWC) previously introduced. Unlike PWC, however, our new progressive transform coder (PTC) does not use wavelets; it performs the space-frequency decomposition step via a new lapped biorthogonal transform (LBT). PTC achieves a rate distortion performance that is comparable (within 2%) to that of the state-of-the-art SPIHT (set partitioning in hierarchical trees) codec. However, thanks to the use of the LBT, the space-frequency decomposition step in PTC reduces the number of multiplications per pixel by a factor of 2.7, and the number of additions by about 15%, when compared to the fastest possible implementation of the "9/7" wavelet transform via lifting. Furthermore, since most of the computation in the LBT is in fact performed by a DCT, our PTC codec can make full use of fast software and hardware modules for 1D and 2D DCT.

Fast and efficient image compression can be achieved with the progressive wavelet coder (PWC) introduced. Unlike many previous wavelet coders, PWC does not rely on zerotrees or other ordering schemes based on parent-child wavelet relationships. PWC has a very simple structure, based on two key concepts: (1) data-independent reordering and blocking, and (2) low-complexity independent encoding of each block via adaptive Rice coding of bit planes. In that way, PWC allows for progressive image encoding that is scalable both in resolution and bit rate, with a fully embedded bitstream. PWC achieves a rate/distortion performance that is comparable to that of the state-of-the-art SPIHT (set partitioning in hierarchical trees) coder, but with a better performance/complexity ratio.

This paper introduces a new structure for a modulated complex lapped transform (MCLT), which is a complex extension of the modulated lapped transform (MLT). The MCLT is a particular kind of a 2/spl times/ oversampled generalized DFT filter bank, whose real part corresponds to the MLT. That property can be used for efficient implementation of joint echo cancellation, noise reduction, and coding, for example. Fast algorithms for the MCLT are presented, as well as examples that show the good performance of the MCLT in noise reduction and echo cancellation.

We have created a system for capturing both the three-dimensional geometry and color and shading information for human facial expressions.We use this data to reconstruct photorealistic, 3D animations of the captured expressions.The system uses a large set of sampling points on the face to accurately track the three dimensional deformations of the face.Simultaneously with the tracking of the geometric data, we capture multiple high resolution, registered video images of the face.These images are used to create a texture map sequence for a three dimensional polygonal face model which can then be rendered on standard 3D graphics hardware.The resulting facial animation is surprisingly life-like and looks very much like the original live performance.Separating the capture of the geometry from the texture images eliminates much of the variance in the image data due to motion, which increases compression ratios.Although the primary emphasis of our work is not compression we have investigated the use of a novel method to compress the geometric data based on principal components analysis.The texture sequence is compressed using an MPEG4 video codec.Animations reconstructed from 512x512 pixel textures look good at data rates as low as 240 Kbits per second.

Two new lapped transforms are introduced: the LBT (lapped biorthogonal transform) and the HLBT (hierarchical lapped biorthogonal transform). The LBT has the same computational complexity of the LOT (lapped orthogonal transform), with much less blocking artifacts. The HLBT has a significantly lower computational complexity than the LOT, essentially no blocking artifacts, and less ringing artifacts than the commonly-used DCT (discrete cosine transform). The LBT and HLBT have a transform coding gain that is typically between 0.5 and 2.5 dB higher than that of the DCT. Image coding examples using JPEG and embedded zerotree coders demonstrate the better performance of the LBT and HLBT.

The blocking artifacts that arise in the use of traditional vector quantization (VQ) schemes can, in general, be virtually eliminated via an efficient lapped VQ strategy. With lapped VQ, blocks are obtained from the source in an overlapped manner, and reconstructed via superposition of overlapped codevectors. The new scheme, which we term lapped orthogonal vector quantization (LOVQ), requires no increase in bit rate and, in contrast to other proposed approaches, no significant increase in computational complexity or memory requirements. Attractively, the use of LOVQ also leads to a modest increase in coding gain over traditional VQ schemes of comparable complexity.

The use of the extended lapped transform (ELT) to implement orthogonal wavelet transforms (WTs) is suggested, and a fast algorithm is presented for the ELT-based WT. Its performance is quite close to that of maximally-regular WTs, in terms of regularity, filtering characteristics, and coding gains. The advantage of ELT-based WTs is that they lead to about half the computational complexity of maximally-regular WTs.>

The author presents a fast algorithm for extended lapped transform (ELT), which is a modulated lapped transform (MLT) with longer basis functions. The proposed algorithm is based on a type-IV discrete cosine transform (DCT-IV). For AR (autoregressive) signals and also for real speech signals, the coding performance of the ELT is shown to be significantly higher than that of a block transform such as the DCT (discrete cosine transform), actually approaches the performance of ideal filter banks. Therefore, the ELT is a promising substitute for traditional block transforms in transform coding systems, and also a good substitute for less efficient filter banks in subband coding systems.>

The hierarchical lapped transform (HLT) is defined. It is based on the modulated lapped transform (MLT). The HLT has a much lower computational complexity than a tree-structured QMF filter bank, and so with HLTs a much larger number of bands can be used in practice. The coding gain of HLTs is close to that of a full-length MLT, for the same number of bands, and therefore there is no significant loss of coding efficiency. With HLTs transient signals can be better reconstructed than with nonhierarchical transforms, as demonstrated by speech and image coding examples. In image coding applications, the HLT can also be used for progressive transmission.>

A new derivation for the lapped orthogonal transform (LOT) is presented. It has basis functions that overlap adjacent blocks and decay smoothly to zero, and so it can reduce the blocking effect to very low levels, without any data overhead. It is shown that a quasioptimal LOT can be computed with an algorithm based on the discrete cosine transform (DCT), with only 30% more computation than the DCT. Unlike earlier approaches to the reduction of blocking effects, there is no penalty in coding gain for the LOT, which actually increases the coding gain by 0.25 dB for a first-order Markov process with rho =0.95.>

A technique for the design of finite-impulse-response (FIR) filters for decimation and interpolation in multirate systems is introduced. FIR prefilters and postfilters that jointly minimize a frequency-weighted mean-square error between the original and reconstructed signals can be designed. There is no need for ideal filter prototypes: the optimal pre-postfilter pair is determined from the signal and noise spectra and the up-sampling and down-sampling factors. Some examples of image and speech processing show that the mean square optimal filter pair leads to typical SNR improvements of 2-6 dB, in comparison to other commonly used filters.>