Thanks to the increasing number of images stored in the cloud, external image similarities can be leveraged to efficiently compress images by exploiting inter-images correlations. In this paper, we propose a novel image prediction scheme for cloud storage. Unlike current state-of-the-art methods, we use a semi-local approach to exploit inter-image correlation. The reference image is first segmented into multiple planar regions determined from matched local features and super-pixels. The geometric and photometric disparities between the matched regions of the reference image and the current image are then compensated. Finally, multiple references are generated from the estimated compensation models and organized in a pseudo-sequence to differentially encode the input image using classical video coding tools. Experimental results demonstrate that the proposed approach yields significant rate-distortion performance improvements compared with the current image inter-coding solutions such as high efficiency video coding.

This paper describes a light field scalable compression scheme based on the sparsity of the angular Fourier transform of the light field. A subset of sub-aperture images (or views) is compressed using HEVC as a base layer and transmitted to the decoder. An entire light field is reconstructed from this view subset using a method exploiting the sparsity of the light field in the continuous Fourier domain. The reconstructed light field is enhanced using a patch-based restoration method. Then, restored samples are used to predict original ones, in a SHVC-based SNR-scalable scheme. Experiments with different datasets show a significant bit rate reduction of up to 24% in favor of the proposed compression method compared with a direct encoding of all the views with HEVC. The impact of the compression on the quality of the all-in-focus images is also analyzed showing the advantage of the proposed scheme.

In this paper, we propose a novel scheme for scalable image coding based on the concept of epitome. An epitome can be seen as a factorized representation of an image. Focusing on spatial scalability, the enhancement layer of the proposed scheme contains only the epitome of the input image. The pixels of the enhancement layer not contained in the epitome are then restored using two approaches inspired from local learning-based super-resolution methods. In the first method, a locally linear embedding model is learned on base layer patches and then applied to the corresponding epitome patches to reconstruct the enhancement layer. The second approach learns linear mappings between pairs of co-located base layer and epitome patches. Experiments have shown that the significant improvement of the rate-distortion performances can be achieved compared with the Scalable extension of HEVC (SHVC).

Thanks to the increasing number of images stored in the cloud, external image redundancies can be leveraged to efficiently compress images by exploiting inter-images correlations. In this paper, we propose a novel cloud-based image coding scheme. Unlike current state-of-the-art systems, our method relies on a data dimensionality reduction technique. A global compensation is associated to a locally-weighted template matching compensation method to predict a reference frame, to be then differential-coded with classic video coding tools. Experimental results demonstrate that the proposed approach yields significant rate-distortion performance improvements compared to current image coding solutions.

This paper describes a novel scheme to reduce the quantization noise of compressed videos and improve the overall coding performances. The proposed scheme first consists in clustering noisy patches of the compressed sequence. Then, at the encoder side, linear mappings are learned for each cluster between the noisy patches and the corresponding source patches. The linear mappings are then transmitted to the decoder where they can be applied to perform de-noising. The method has been tested with the HEVC standard, leading to a bitrate saving of up to 9.63%.

This paper presents a color inter-layer prediction (ILP) method for scalable coding of high dynamic range (HDR) video content with a low dynamic range (LDR) base layer. Relying on the assumption of hue preservation between the colors of an HDR image and its LDR tone mapped version, we derived equations for predicting the chromatic components of the HDR layer given the decoded LDR layer. Two color representations are studied. In a first encoding scheme, the HDR image is represented in the classical Y'CbCr format. In addition, a second scheme is proposed using a colorspace based on the CIE u'v' uniform chromaticity scale diagram. In each case, different prediction equations are derived based on a color model ensuring the hue preservation. Our experiments highlight several advantages of using a CIE u'v'-based colorspace for the compression of HDR content, especially in a scalable context. In addition, our ILP scheme using this color representation improves on the state-of-the-art ILP method, which directly predicts the HDR layer u'v' components by computing the LDR layers u'v' values of each pixel.

This paper presents a scalable high dynamic range (HDR) image coding framework in which the base layer is a low dynamic range (LDR) version of the image that may have been generated by an arbitrary Tone Mapping Operator (TMO). Our method successfully handles the case of complex local TMOs thanks to a block-wise and non-linear approach. A novel template based Inter Layer Prediction (ILP) is designed in order to perform the inverse tone mapping of a block without the need to transmit any additional parameter to the decoder. This method enables the use of a more accurate inverse tone mapping model than the simple linear regression commonly used for block-wise ILP. Our experiments have shown an average bitrate saving of 34% on the HDR enhancement layer, compared to state of the art methods.

We describe a novel epitomic image representation scheme that factors a given image content into a condensed epitome and a low-resolution image to reduce the memory space for images. Given an input image, we construct a condensed epitome such that all image patches can successfully be reconstructed from the factored representation by means of an optimized neighbor-embedding strategy. Under this new scope of epitomic image representations aligned with the manifold sampling assumption, we end up a more generic epitome learning scheme with increased optimality, compactness, and reconstruction stability. We present the performance of the proposed method for image and video up-scaling (super-resolution) while extensions to other image and video processing are straightforward.

This paper presents a scalable high dynamic range (HDR) image coding scheme in which the base layer is a low dynamic range version of the image that may have been generated by an arbitrary tone mapping operator (TMO). No restriction is imposed on the TMO, which can be either global or local, so as to fully respect the artistic intent of the producer. Our method successfully handles the case of complex local TMOs thanks to a block-wise and non-linear approach. A novel template-based interlayer prediction (ILP) is designed in order to perform the inverse tone mapping of a block without the need to transmit any additional parameter to the decoder. This method enables the use of a more accurate inverse tone mapping model than the simple linear regression commonly used for block-wise ILP. In addition, this paper shows that a linear adjustment of the initially predicted block can further improve the overall coding performance by using an efficient encoding scheme of the scaling parameters. Our experiments have shown an average bitrate saving of 47% on the HDR enhancement layer, compared with the previous local ILP methods.

High Dynamic Range (HDR) images contain more intensity levels than traditional image formats. Instead of 8 or 10 bit integers, floating point values are generally used to represent the pixel data. To extend the use of existing video codecs such as HEVC to HDR floating point video sequences, we propose a method that converts the floating point data and reduces the bit depth of input images with minimal loss. Several variants of the method are proposed. They are adapted to different quality requirements. In particular, near lossless compression is addressed.

This paper describes a novel image coding scheme based on epitome inpainting. An epitome containing a factorized texture representation of the image is first coded and transmitted. The decoded epitome is then inpainted by propagating its structure and texture with extended H.264 Intra directional prediction modes and advanced neighbor embedding methods. The blocks to be inpainted are processed according to an adaptive scanning order which adapts to the image content and gives a higher priority to the blocks having strong structures. The inpainting step has been incorporated in the coding loop of an H.264 encoder, leading to a bitrate saving of up to 48.12% with respect to H.264 intra.

We propose an exemplar-based super-resolution algorithm based on sparsity constrained neighbor-embeddings of local image patches. We extract exemplar patch pairs from as little as the given low-resolution image, and we rely on local geometric similarities of low-and high-resolution patch spaces. While sparsely coding the local geometry with a greedy patch selection method, we refine our solution by iteratively updating the obtained high-resolution image. We finally apply an adaptive back-projection to ensure the global consistency. Our experimental results indicate promising performance on synthesizing natural looking textures and sharp edges when compared to other super-resolution methods from the literature.

Tone Mapping Operators (TMOs) transform High Dynamic Range (HDR) contents to address Low Dynamic Range (LDR) displays. However, before reaching the end-user, these contents are usually compressed using a codec (coder-decoder) for broadcasting or storage purposes. Achieving the best trade-off between rendering and compression efficiency is of prime importance. Any TMO includes a rounding quantization to convert floating point values to integer ones. In this work, we propose to modify this quantization to increase the compression efficiency of the tone mapped content. By using a motion compensation, our technique preserves the rendering intent of the TMO while maximizing the correlations between successive frames. Experimental results show that we can save up to 12% of the total bit-rate as well as an average bit-rate reduction of 8.5% for all the test sequences. We show that our technique can be applied to other applications such as denoising.

The paper describes a K-NN search method aided by local learning of subspace mappings for the problem of neighbor-embedding based image Intra prediction. The local learning of subspace mappings relies on multivariate linear regression. The method is used jointly with Locally Linear Embedding (LLE) as well as with a method inspired from Non Local Means (NLM) for prediction. Linear and kernel ridge regression are also considered directly for predicting the unknown pixels. Rate-distortion performances are then given in comparison with Intra prediction using LLE and classical K-NN search, as well as in comparison with H.264 Intra prediction modes.

Image prediction methods based on data dimensionality reduction techniques have been recently introduced for still images. These techniques have been proven efficient, especially when used in a H.264 framework. This paper introduces a natural extension of this work from spatial prediction to temporal prediction. Locally Linear Embedding and Optimized Map-Aided Locally Linear Embedding methods are adapted within H.264 in order to improve inter image prediction. The resulting prediction methods are compared with the H.264 motion estimation/compensation method, template matching methods and Adaptive Interpolation Filters, and are shown to bring significant Rate-Distortion performance improvements (up to 20.21 % rate saving with respect to H.264).

We describe a self-content single-image super-resolution algorithm based on multi-scale neighbor embeddings of small image patches. Given an input low-resolution patch, we gradually expand its size by relying on local geometric similarities of low- and high-resolution patch spaces under small scaling factors. We characterize the local geometry with K-similar patches taken from an exemplar set and we collect exemplar patch pairs from the input image and its appropriately rescaled versions. While ensuring local images compatibility with an optimization on K, we satisfy image smoothness by patch overlapping. We further enforce global consistency through an adaptive back-projection. Our experimental results show better performance on synthesizing natural looking textures and sharp edges with less artifacts when compared to other methods.

High Dynamic Range (HDR) is a strong candidate for the next video format, closer to the human perception for an increased user experience. However, the HDR video compression field is still under much investigation. Most of the time, HDR video codecs are based on already existing distribution-dedicated LDR video codecs, for instance H.264/AVC or H.265/HEVC. Some coding methods use two instances of a LDR codec: one for the LDR layer and one for the HDR residual layer. Other methods encode HDR frames directly using a high profile professional codec with a bit-depth well beyond the standard 8 bits. In this paper, we present a technique that requires only one instance of an 8-bit distribution codec. Our technique transforms HDR frames into LDR-like ones and uses metadatas to help the decoder to achieve a good HDR reconstruction. Results show that our solution performs well for bit-rate targeted in broadcast video content.

This paper describes new image prediction methods based on neighbor embedding (NE) techniques. Neighbor embedding methods are used here to approximate an input block (the block to be predicted) in the image as a linear combination of K nearest neighbors. However, in order for the decoder to proceed similarly, the K nearest neighbors are found by computing distances between the known pixels in a causal neighborhood (called template) of the input block and the co-located pixels in candidate patches taken from a causal window. Similarly, the weights used for the linear approximation are computed in order to best approximate the template pixels. Although efficient, these methods suffer from limitations when the template and the block to be predicted are not correlated, e.g., in non homogenous texture areas. To cope with these limitations, this paper introduces new image prediction methods based on NE techniques in which the K-NN search is done in two steps and aided, at the decoder, by a block correspondence map, hence the name map-aided neighbor embedding (MANE) method. Another optimized variant of this approach, called oMANE method, is also studied. In these methods, several alternatives have also been proposed for the K-NN search. The resulting prediction methods are shown to bring significant rate-distortion performance improvements when compared to H.264 Intra prediction modes (up to 44.75% rate saving at low bit rates).

Template matching has been shown to outperform the H.264 prediction modes for Intra video coding thanks to better spatial prediction and no additional ancillary data to transmit. The method indeed works well when the template and the block to be predicted are highly correlated, e.g., in homogenous image areas, however, it obviously fails in areas with non homogeneous textures. This paper explores the idea of using a block-matching intra prediction algorithm which, thanks to a Rate-Distorsion (RD) based decision mechanism, will naturally be used in image areas when template matching (TM) fails. This new method offers a significant coding gain compared to H.264 Intra prediction modes and the template matching based prediction. Indeed, the TM-based algorithm and the proposed hybrid algorithm lead, with the Bjontergaard measure, to rate gains of up to respectively 38.02% and 48.38% at low bitrates when compared with H.264 Intra only.

Image prediction methods based on data dimensionality reduction techniques have been introduced in [1]. Although efficient, these methods suffer from limitations when the block to be predicted and its neighborhood (or template) are not correlated, e.g. in non homogenous texture areas. To cope with these limitations, this paper introduces new image prediction methods based on locally linear embedding (LLE) technique in which the required K-NN search is aided, at the decoder, by a block correspondence map, hence the name Map-Aided Locally Linear Embedding (MALLE) method. Another optimized variant of this approach, called oMALLE method, is also studied. The resulting prediction methods are shown to bring significant Rate-Distortion (RD) performance improvements when compared to H.264 Intra prediction modes (up to 40.78 % rate saving at low bit rates).

State-of-the-art video compression technologies, such as MPEG-4 AVC/H.264 or the new HEVC standard being developed by ISO MPEG and ITU-T VCEG, make use of tree-structured block partitioning for motion compensation. Such motion partitioning only captures horizontal and vertical motion boundaries. To better match actual motion frontiers, Geometry adaptive block partitioning (GEO) has been explored for several years. GEO enables splitting a block using non-horizontal or non-vertical line. Although noticeable coding efficiency gains can be obtained, GEO involves a significant increase of the number of modes to be tested with therefore a high impact on encoding complexity. This paper presents fast algorithms aiming at controlling the complexity while saving the coding efficiency gains of GEO. Experimental results are provided on top of the HEVC standard, demonstrating first the efficiency of the GEO tool. Simplified versions, offering noticeable complexity reduction with limited rate-distortion performance loss, are also demonstrated and compared.

This paper addresses the problem of epitome construction for image compression. An optimized epitome construction method is first described, where the epitome and the associated image reconstruction, are both successively performed at full pel and sub-pel accuracy. The resulting complete still image compression scheme is then discussed with details on some innovative tools. The PSNR-rate performance achieved with this epitome-based compression method is significantly higher than the one obtained with H.264 Intra and with state of the art epitome construction method. A bit-rate saving up to 16% comparatively to H.264 Intra is achieved.

In this paper, we propose a new intra prediction scheme based on hard thresholding applied on shift-DCT coefficients. The explored intra prediction method, validated in an H.264/AVC framework, aims at better predicting complex 2D patterns that cannot be properly extrapolated using H.264/AVC directional intra prediction modes. The basic explored algorithm makes use of neighbor pixels of the current block to predict the current signal using a shift-DCT based extrapolation. Different versions of the algorithm, corresponding to different hard thresholding parameter derivation processes, are described. Experimental results show that the proposed intra coding prediction method can significantly improve the coding efficiency compared to the directional predictive modes of H.264/AVC, with an average BD-rate gain of 6.2% observed on a various pictures test set.

This work, we propose the use of sparse signal representation techniques to solve the problem of closed-loop spatial image prediction. The reconstruction of signal in the block to predict is based on basis functions selected with the matching pursuit (MP) iterative algorithm, to best match a causal neighborhood. We evaluate this new method in terms of PSNR and bitrate in a H.264/AVC encoder. Experimental results indicate an improvement of rate-distortion performance. In this paper, we also present results concerning the use of this technique for intra-inter layer prediction refinement, in a scalable video coding (SVC) like scheme.

Visual attention is a mechanism which filters out redundant visual information and detects the most relevant parts of our visual field. Automatic determination of the most visually relevant areas would be useful in many applications such as image and video coding, watermarking, video browsing, and quality assessment. Many research groups are currently investigating computational modeling of the visual attention system. The first published computational models have been based on some basic and well-understood Human Visual System (HVS) properties. These models feature a single perceptual layer that simulates only one aspect of the visual system. More recent models integrate complex features of the HVS and simulate hierarchical perceptual representation of the visual input. The bottom-up mechanism is the most occurring feature found in modern models. This mechanism refers to involuntary attention (i.e., salient spatial visual features that effortlessly or involuntary attract our attention). This paper presents a coherent computational approach to the modeling of the bottom-up visual attention. This model is mainly based on the current understanding of the HVS behavior. Contrast sensitivity functions, perceptual decomposition, visual masking, and center-surround interactions are some of the features implemented in this model. The performances of this algorithm are assessed by using natural images and experimental measurements from an eye-tracking system. Two adequate well-known metrics (correlation coefficient and Kullbacl-Leibler divergence) are used to validate this model. A further metric is also defined. The results from this model are finally compared to those from a reference bottom-up model.

A new spatio-temporal model for simulating the bottom-up visual attention is proposed. It has been built from numerous important properties of the human visual system (HVS). This paper focuses both on the architecture of the model and on its performances. Given that the spatial model of the bottom-up visual attention has already been defined [O. Le Meur et al., 2004], the temporal dimension is more accurately described. A qualitative and quantitative comparison with human fixations collected from an eye tracking apparatus is undertaken. From the former, the quality of the prediction is deemed very good whereas the latter illustrates that the best predictor of the human fixation consists of the sum all visual features (achromatic, chromatic and motion).

It is now commonly assumed that the human visual attention, which is a selecting process of the most relevant locations in a scene according to a particular behavior, is driven by both top-down (task-dependent) and bottom-up (signal-dependent) control. A new model attempting to simulate the bottom-up process has been designed Le Meur, O et al., (2004). This model is purely based on visual system properties that provides noticeable advantages compared to the classical published approaches. This paper focuses on the performance assessment of this model by achieving a comparison with real fixation points stemming from eye-tracking apparatus both subjectively and objectively.

This paper presents an implementation of an end-to-end application for streaming stored MPEG-4 fine-grained scalable (FGS) videos over the best-effort Internet. Our current implementation runs over TCP but can also be run over TCP-friendly RTP/UDP with an error recovery mechanism. Our scheme adapts the coding rate of the streaming video to the variations of the available bandwidth for the connection, while smoothing changes in image quality between consecutive video scenes. We justify our design choices and present the results of the simulations obtained from our testbed. Our combined use of FGS-encoded video with a simple network-adaptation heuristic gives a system of low complexity, which is suitable for high performing video servers.