Theplenoptic functiondescribes the visual information available to an observer at any point in space and time. Samples of the plenoptic function (POF) are seen in video and in general visual content (images, mosaics, panoramic scenes, etc.), and represent large amounts of information. In this paper, we propose a stochastic model to study the compression limits of a simplified version of the plenoptic function. In the proposed framework, we isolate the two fundamental sources of information in the POF: the one representing the camera motion and the other representing the information complexity of the ¿reality¿ being acquired and transmitted. The sources of information are combined, generating a stochastic process that we study in detail. We first propose a model for ensembles of realities that do not change over time. The proposed model is simple in that it enables us to derive precise coding bounds in the information-theoretic sense that are sharp in a number of cases of practical interest. For this simple case of static realities and camera motion, our results indicate that coding practice is in accordance with optimal coding from an information-theoretic standpoint. The model is further extended to account for visual realities that change over time. We derive bounds on the lossless and lossy information rates for this dynamic reality model, stating conditions under which the bounds are tight. Examples with synthetic sources suggest that within our proposed model, common hybrid coding using motion/displacement estimation with DPCM performs considerably suboptimally relative to the true rate-distortion bound.

We propose a stochastic model for video and compute its information rates. The model has two sources of information representing ensembles of camera motion and visual scene data (i.e. "realities"). The sources of information are combined generating a vector process that we study in detail. Both lossless and lossy information rates are derived. The model is further extended to account for realities that change over time. We derive bounds on the lossless and lossy information rates for this dynamic reality model, stating conditions under which the bounds are tight. Experiments with synthetic sources suggest that in the presence of scene motion, simple hybrid coding using motion estimation with DPCM can be suboptimal relative to the true rate-distortion bound

The contourlet transform was proposed to address the limited directional resolution of the separable wavelet transform. One way to guarantee good approximation behavior is to let the directional filters in the contourlet filter bank have sharp frequency response. This requires filters with large support size. We seek to isolate the key filter property that ensures good approximation. In this direction, we propose filters with directional vanishing moments (DVM). These filters, we show, annihilate information along a given direction. We study two-channel filter banks with DVM filters. We provide conditions under which the design of DVM filter banks is possible. A complete characterization of the product filter is, thus, obtained. We propose a design framework that avoids 2-D factorization using the mapping technique. The filters designed, when used in the contourlet transform, exhibit nonlinear approximation comparable to the conventional filters while being shorter and, therefore, providing better visual quality with less ringing artifacts. Furthermore, experiments show that the proposed filters outperform the conventional ones in image approximation and denoising.

We study the compression problem of visual scenes acquired with a camera for transmission or storage. Our proposed model is general and includes two well-known cases: that of video coding and that of lightfield data compression. Those two examples are related in that both are characterized by two sources of complexity: the camera motion, and the scenes being acquired. The main difference of the two is in how the complexity of the camera is coded. We propose a simplified model which includes the main characteristics of the general problem. Based on this, we do theoretical analysis, develop simple codes, and show experimental results.

In this paper we propose a geometrical entropy coder to compress images with geometry. Our proposed scheme works on quantized coefficients of the discrete wavelet transform (DWT). The geometry inherent in those coefficients is exploited by means of directional, predictive coding. We accomplish directional prediction with the aid of the coefficients of the undecimated wavelet transform (UDWT), which are estimated from the available DWT ones both at the encoder and decoder. Following directional prediction, a simple entropy coder takes care of the remaining redundancy. The resulting codec attains competitive performance.

In this paper, we develop the nonsubsampled contourlet transform (NSCT) and study its applications. The construction proposed in this paper is based on a nonsubsampled pyramid structure and nonsubsampled directional filter banks. The result is a flexible multiscale, multidirection, and shift-invariant image decomposition that can be efficiently implemented via the à trous algorithm. At the core of the proposed scheme is the nonseparable two-channel nonsubsampled filter bank (NSFB). We exploit the less stringent design condition of the NSFB to design filters that lead to a NSCT with better frequency selectivity and regularity when compared to the contourlet transform. We propose a design framework based on the mapping approach, that allows for a fast implementation based on a lifting or ladder structure, and only uses one-dimensional filtering in some cases. In addition, our design ensures that the corresponding frame elements are regular, symmetric, and the frame is close to a tight one. We assess the performance of the NSCT in image denoising and enhancement applications. In both applications the NSCT compares favorably to other existing methods in the literature.

In this paper we study 2D nonseparable filter banks that annihilate information along a certain discrete direction. This is done by having filters with directional vanishing moments (DVM). We study the approximation property of such filters and the design problem providing conditions for its solvability. In particular, we completely characterize the solution and propose a design procedure utilizing the mapping technique. Nonlinear approximation experiments with the contourlet transform indicate that compared with the traditional filters, the new filters designed with DVM provide gains in SNR and visual quality due to their short size.

In this paper we study the nonsubsampled contourlet transform. We address the corresponding filter design problem using the Mc-Clellan transformation. We show how zeroes can be imposed in the filters so that the iterated structure produces regular basis functions. The proposed design framework yields filters that can be implemented efficiently through a lifting factorization. We apply the constructed transform in image noise removal where the results obtained are comparable to the state-of-the art, being superior in some cases.

We present the nonsubsampled contourlet transform and its application in image enhancement. The nonsubsampled contourlet transform is built upon nonsubsampled pyramids and nonsubsampled directional filter banks and provides a shift-invariant directional multiresolution image representation. Existing methods for image enhancement cannot capture the geometric information of images and tend to amplify noises when they are applied to noisy images since they cannot distinguish noises from weak edges. In contrast, the nonsubsampled contourlet transform extracts the geometric information of images, which can be used to distinguish noises from weak edges. Experimental results show the proposed method achieves better enhancement results than a wavelet-based image enhancement method.