The direction-adaptive partitioned block transform (DA-PBT) is proposed to exploit the directional features in color images to improve coding performance. Depending on the directionality in an image block, the transform either selects one of the eight directional modes or falls back to the nondirectional mode equivalent to the conventional 2-D DCT. The selection of a directional mode determines the transform direction that provides directional basis functions, the block partitioning that spatially confines the high-frequency energy, the scanning order that arranges the transform coefficients into a 1-D sequence for efficient entropy coding, and the quantization matrix optimized for visual quality. The DA-PBT can be incorporated into image coding using a rate-distortion optimized framework for direction selection, and can therefore be viewed as a generalization of variable blocksize transforms with the inclusion of directional transforms and nonrectangular partitions. As a block transform, it can naturally be combined with block-based intra or inter prediction to exploit the directionality remaining in the residual. Experimental results show that the proposed DA-PBT outperforms the 2-D DCT by more than 2 dB for test images with directional features. It also greatly reduces the ringing and checkerboard artifacts typically observed around directional features in images. The DA-PBT also consistently outperforms a previously proposed directional DCT. When combined with directional prediction, gains are less than additive, as similar signal properties are exploited by the prediction and the transform. For hybrid video coding, significant gains are shown for intra coding, but not for encoding the residual after accurate motion-compensated prediction.

Local features are widely used for content-based image retrieval and object recognition. We present an efficient method for encoding digital images suitable for local feature extraction. First, we find the patches in the image corresponding to the detected features. Then, we extract these patches at their characteristic scale and orientation and encode them for efficient transmission. A Discrete Cosine Transform (DCT) with adaptive block size is used for patch compression. We compare this method to directly compressing feature descriptors using transform coding. Experimental results show the superior performance of our technique. Image patches can be compressed to rates around 55 bits/patch (18times compression relative to uncompressed SIFT feature descriptors) and still achieve good image matching performance.

We propose a direction-adaptive block transform that exploits the directionality in images to improve the compression performance. It can be viewed as a generalization of variable block-size transforms with the inclusion of non-rectangular partitions and directional transforms. As a block transform, it can be directly combined with block-wise intra and inter prediction in video coding standards. Experimental results show that the proposed transform outperforms the variable block-size 2-D DCT by up to 3 dB. It also outperforms a previously proposed direction-adaptive DCT by up to 2 dB. When combined with intra prediction, the proposed transform further provides a gain of 0.5 to 2 dB upon intra prediction with the 2-D DCT.

The direction-adaptive discrete wavelet transform (DA-DWT) locally adapts the filtering direction to the geometric flow in the image. DA-DWT image coders have been shown to achieve a rate-distortion performance superior to non-adaptive wavelet coders. However, since the direction information must always be signalled regardless of total bit-rate, performance at very low bit-rates might be worse. In this paper, we propose two scalable direction representations: the layered scheme which is similar to the scalable motion vector representation in scalable video coding and the level-unit scheme which provides finer granularity upon the layered scheme. Experimental results indicate that we can achieve the desirable performance at both low and high bit rates with our proposed level-unit scheme. Significant improvement in image quality (about 3-5 dB) is observed at very low bit rate, relative to non-scalable coding of the direction information.

We propose a direction-adaptive DWT (DA-DWT) that locally adapts the filtering directions to image content based on directional lifting. With the adaptive transform, energy compaction is improved for sharp image features. A mathematical analysis based on an anisotropic statistical image model is presented to quantify the theoretical gain achieved by adapting the filtering directions. The analysis indicates that the proposed DA-DWT is more effective than other lifting-based approaches. Experimental results report a gain of up to 2.5 dB in PSNR over the conventional DWT for typical test images. Subjectively, the reconstruction from the DA-DWT better represents the structure in the image and is visually more pleasing.

We propose an adaptive lifted discrete wavelet transform to locally adapt the filtering direction to the geometric flow in the image. The proposed approach refines previous directional lifting approaches to achieve superior compression performance with less demand for computation. Additionally, a bandeletization procedure is combined with directional lifting in a unified framework to further remove the correlation in the wavelet coefficients. Up to 2.8 dB improvement in PSNR over the conventional 2-D CDF 9/7 wavelet transform for natural images is reported. Significant improvement in subjective quality is also observed.

We propose disparity-compensated lifting for wavelet compression of light fields. With this approach, we obtain the benefits of wavelet coding, such as scalability in all dimensions, as well as superior compression performance. Additionally, the proposed approach solves the irreversibility limitations of previous light field wavelet coding approaches, using the lifting structure. Our scheme incorporates disparity compensation into the lifting structure for the transform across the views in the light field data set. Another transform is performed to exploit the coherence among neighboring pixels, followed by a modified SPIHT coder and rate-distortion optimized bitstream assembly. A view-sequencing algorithm is developed to organize the views for encoding. For light fields of an object, we propose to use shape adaptation to improve the compression efficiency and visual quality of the images. The necessary shape information is efficiently coded based on prediction from the existing geometry model. Experimental results show that the proposed scheme exhibits superior compression performance over existing light field compression techniques.

We propose a novel adaptive wavelet transform that exploits local image properties for image compression. It combines wavelet filters adaptive to edge orientations with quincunx subsampling to form a 2-D nonseparable transform through lifting. Filter selections are efficiently represented. Significant improvement on both subjective and objective quality over the conventional separable transform is observed. In addition, unlike previous adaptive transforms, the symmetry in quincunx subsampling enables even quality for image features along different directions and the compression performance is insensitive to image orientation

In wavelet video coding, the quantization error in the temporal sub-bands propagates through temporal wavelet synthesis to the reconstructed frames. We analyze this propagation by considering the impact of the spatial interpolation in motion compensation on the statistics of the quantization error. Based on the analysis, a distortion model that can be derived from the motion fields is introduced and incorporated into a rate allocation method aiming at reducing the quality fluctuation over frames. Comparing to the conventional allocation method that ignores the effect of motion compensation, significant reduction in quality fluctuation is observed while incurring minimal loss in the average quality

In wavelet video coding, due to motion-compensation entwined in the temporal wavelet transform, the distortion in the temporal subbands propagates in an uneven manner into the reconstructed frames of a group of pictures. This leads to fluctuation of reconstruction quality in time, which can be visually displeasing. We propose a new update step which is derived by taking into account a Lagrangian term for even distribution of distortion among reconstructed frames in addition to minimizing total distortion. Additionally some heuristics for the implementation of this new update step are proposed. Experimental results show reduction of quality fluctuation compared to the conventional update step

We propose a rate-distortion optimized framework to stream scalable bitstreams of 3-D wavelet video stored at the sender to a remote receiver. Based on the source rate-distortion profiles, the desired playout deadline, transmission rate, and the network characteristics, the receiver issues customized requests throughout the video playout session in order to retrieve the data that minimize the distortion in the reconstructed frames. Rate-distortion optimized data request is formulated as a convex optimization problem. Experimental results show that the proposed scheme improves the video quality by up to 3.1 dB over the heuristic scheme at the same rate, or correspondingly reduces the required the transmission rate by up to 60% for the same quality.

The paper proposes a receiver-based rate-distortion optimized framework to stream interactively scalable bitstreams of light fields. Based on the desired viewpoints, the predicted future buffer state, the network characteristics, and the target transmission rate, the receiver customizes the data request in order to minimize the distortion in the frames to be rendered. Experimental results show that the proposed framework improves the rendering quality by 1.0-2.8 dB over a heuristic scheme at the same rate. Correspondingly, to achieve the same rendering quality the proposed framework reduces 25-35% of the rate over the heuristic scheme.

We propose a sender-based rate-distortion optimized framework to stream scalable bitstreams of 3-D wavelet video stored at the sender to a remote receiver. Based on the requests and feedback from the receiver, the source rate-distortion profiles, the desired playout latency and transmission rate, and the network characteristics, the sender optimizes the responses sent to the receiver throughout the video playout session in order to minimize the distortion in the reconstructed frames. Rate-distortion optimized response is formulated as a convex optimization problem, and an offline-computation approach is proposed to further reduce the complexity at the sender. Experimental results show that the proposed sender-based approach outperform the receiver-based approach, and the offline-computation approach very closely approximates the fully-optimized approach.

A rate-distortion optimized scheme for interactive light field streaming is proposed. The light field data set is transformed into blocks of wavelet coefficients; each block is coded as a scalable bitstream and stored at the sender. To render a frame, the receiver issues a request for relevant data. Based on the request, the estimated state of the data already at the receiver, the network characteristics, and the desired transmission rate, the sender customizes the outgoing packets in order to minimize the distortion experienced at receiver. Experimental results show that the proposed rate-distortion optimized scheme improves the rendering quality by 0.5~2.1 dB in PSNR over a heuristic scheme at the same rate. Alternatively, it reduces the required bit-rate by 10%~25% over the heuristic scheme at the same rendering quality.

We propose a novel approach for light field compression that incorporates disparity compensation into 4D wavelet coding using disparity-compensated lifting. With this approach, we obtain the benefits of wavelet coding, including compression efficiency and scalability in all dimensions. Additionally, our proposed approach solves the irreversibility limitations of previous wavelet coding approaches. Experimental results show that the compression efficiency of the proposed technique outperforms current state-of-the-art wavelet coding techniques by a wide margin.

We propose a method of using shape adaptation for compression of light fields of 3D objects. Shape adaptation is incorporated into two light field coders, both applying disparity compensation with an explicit geometry model, to improve the compression efficiency. The shape information can be derived at the decoder from an accurate geometry model. If the available geometry is inaccurate, we propose to code the exact 2D shapes with the aid of the approximate geometry. Experiments show that shape adaptation greatly improves the compression performance of both coders.

We propose a novel approach for light field compression that incorporates disparity compensation into 4-D wavelet coding using disparity-compensated lifting. With this approach, we obtain the benefits of wavelet coding, including compression efficiency and scalability in all dimensions. Additionally, our proposed approach solves the irreversibility limitations of previous wavelet coding approaches. Experimental results show that the compression efficiency of the proposed technique outperforms current state-of-the-art wavelet coding techniques by a wide margin.