This paper presents a novel framework to achieve scalable multiview image compression and view synthesis. The open-loop wavelet-lifting scheme for geometric filtering has been exploited to achieve signal-to-noise ratio scalability and view-type scalability (mono, stereo, or multiview). Spatial scalability is achieved by employing in-band prediction which removes correlations among subbands (level-by-level) via shift-invariant references obtained by overcomplete discrete wavelet transforms. We propose a novel in-band disparity compensated view filtering approach, akin to motion compensated temporal filtering, for achieving a scalable multiview codec. In our codec, hybrid prediction is proposed to deal with occlusions, and a novel cost function in dynamic programming (DP) for disparity estimation is introduced to improve view synthesis quality. Experiments show comparable results at full resolution and significant improvements at coarser resolutions, compared to a conventional spatial prediction scheme. View synthesis efficiency is extensively improved by utilizing disparity estimation from the proposed DP approach.

An unsupervised image compression-by-synthesis system is proposed utilising wavelet based image segmentation and analysis combined with patch based texture synthesis. High perceptual quality is ensured using an artefact detection algorithm in the encoder loop. EBCOT is used to transform code texture samples and residual image data. Resulting bitrate savings of up to approximately 17% over JPEG2000 for little change in perceptual quality have been shown.

This paper presents a novel architecture for the secure delivery of encrypted H.264 SVC bitstreams. It relies on a block cipher and stream cipher used in a novel way that would allow an intermediary transcoder to truncate the bitstream to the appropriate bit-rate without decrypting the data. The system, called SVC-sec, is compared to other architectures presented in the literature and it is shown that SVC-sec offers many benefits, particularly when used with FGS streams.

An image compression scheme is proposed, utilising wavelet- based image segmentation and texture analysis, and patch- based texture synthesis. This has been incorporated into a JPEG framework. Homogeneous textured regions are identified and removed prior to transform coding. These regions are then replaced at the decoder by synthesis from marked samples, and colour matched to ensure similarity to the original. Experimental results on natural images show bitrate savings of over 18% compared with JPEG for little change in measured visual quality.

Optical flow forms an important initial processing stage for many machine vision tasks. A framework is presented for the recovery of dense optical flows from image sequences containing large motions. Sparse feature correspondences are used to assign multiple optical flow hypotheses to each image pixel which are then independently refined to produce a further set of refined hypotheses. One final flow is selected for each pixel from these refined flows by seeking to minimize the local matching error. Dense optical flows from image sequences with small motions are successfully recovered. In image sequences with very large motions, a clear increase in optical flow accuracy is observed when compared to a hierarchical approach to optical flow estimation.

Traditional volumetric scene reconstruction algorithms involve the evaluation of many millions of voxels which is highly time consuming. This paper presents an efficient algorithm based of future frame prediction that can dramatically reduce the number of voxels to be evaluated in time varying scenes. The new prediction method, combining scene flow and morphological dilations, is evaluated against a simple model dilation method. Results show the proposed method outperforms a simple dilation method and has the potential to improve the efficiency of volumetric scene reconstruction algorithms while retaining quality given accurate optical flows. 1

The colour volumetric data which is constructed from a set of multi-view images is capable of providing realistic immersive experience. However it is not widely applicable due to its manifold increase in bandwidth. This paper presents a novel framework to achieve scalable volumetric compression. Based on wavelet transformation, data rearrangement algorithm is proposed to compact volumetric data leading to high efficiency of transformation. The colour data is also rearranged by using the characteristics of human eye sensitivity. Moreover, the pre-processing for adaptive resolution is proposed in this paper. The low resolution overcomes the limitation of the data transmission at low bit rate, whilst the fine resolution improves the synthesised images' quality. The results of our proposed schemes show significant improvement of the compression performance over the traditional 3D coding.

In digital broadcast TV systems, video data is normally encrypted before transmission. For in-home redistribution, it is often necessary to transcode the bitstream to achieve optimum utilization of available bandwidth. If a signal is decrypted before transcoding and re-encrypted, this may lead to a security loophole. This paper presents a solution in the form of a novel H.264 selective encryption algorithm that encrypts sign bits of transform coefficients and motion vectors to allow secure transcoding without decryption. The performance of this system is compared with I-frame encryption. The results show that sign encryption is more secure than I-frame encryption and has a lower complexity. A hybrid system using a modified transcoder and sign encryption is found to give an optimal compromise between security and transcoding performance.

A novel algorithm for disparity/depth estimation from multi-view images is presented. A dynamic programming approach with window-based correlation and a novel cost function is proposed. The smoothness of disparity/depth map is embedded in dynamic programming approach, whilst the window-based correlation increases reliability. The enhancement methods are included, i.e. adaptive window size and shiftable window are used to increase reliability in homogenous areas and to increase sharpness at object boundaries. First, the algorithms estimate depth maps along a single camera axis. The algorithms exploits then combines the depth estimates from different axis to derive a suitable depth map for multi-view images. The proposed scheme outperforms existing approaches in parallel and in the non-parallel camera configurations

In this paper, we propose a novel volumetric representation for a sparse set of calibrated multi-view images of a non-Lambertian scene. The depth map of each reference view is registered into a volume and a simple algorithm to shape the volume is introduced. Particular colours are defined for each voxel to render a smooth and realistic image. Synthesized results demonstrate the good performance with the proposed scheme, both for parallel-camera and non-parallel-camera geometries.

This paper presents a novel approach for synthesizing intermediate or virtual viewpoints (VVs) of a 3D scene based on information from a number of known reference viewpoints (RVs). The proposed approach directly estimates the pixel value (and corresponding depth) for each pixel in the VV. This is contrast to the more traditional 2 stage approach of firstly building a full 3D or 2.5D model for the scene and then synthesising the desired VV. The potential advantage of this approach is that it works directly with the target virtual view and is hopefully less susceptible to the propagation of errors from the depth estimation stage to the interpolation stage.

The main objective of this paper is to develop a fine grained scalable video codec which can fully exploit temporal correlations within all layers, and thus offer performance closer to that of a single layer non-scalable system. To this end we propose the use of a combination of matching pursuits together with absolute value coding of the displaced frame difference information. The proposed codec is also shown to be highly robust to channel errors. This results from a combination of the use of fixed length code-words and a reduction of temporal error propagation. The use of fixed length codewords also facilitates the use of lower complexity transcoding operations to support more sophisticated forms of scalability.

This paper presents a fast matching pursuits algorithm. In addition to a fast implementation, the algorithm also allows the efficient use of non-separable dictionary basis functions. The use of non-separable components allows basis functions which provide a better match to diagonally orientated image features. In order to demonstrate the proposed method, a simple dictionary is developed and used. Even without any sophisticated entropy coding, the proposed system gives performance exceeding that of H.263.

This paper presents a new architecture for designing non-linear, critically decimated, perfect reconstruction filter banks. It is believed that such filters have a wide range of applications including that of still image and image sequence compression. Example filters are presented, for still image compression using a quincunx system. These filters are shown to out-perform other linear and non-linear methods, both in terms of subjective and objective image quality.

This paper examines the use of arithmetic coding in conjunction with the error resilient entropy code (EREC). The constraints on the coding model are discussed and simulation results are presented and compared to those obtained using Huffman coding. These results show that without the EREC, arithmetic coding is less resilient than Huffman coding, while with the EREC both systems yield comparable results.

Many source and data compression schemes work by splitting the input signal into blocks and producing variable-length coded data for each block. If these variable-length blocks are transmitted consecutively, then the resulting coder is highly sensitive to channel errors. Synchronization code words are often used to provide occasional resynchronization at the expense of some added redundant information. This paper introduces the error-resilient entropy code (EREC) as a method for adapting existing schemes to give increased resilience to random and burst errors while maintaining high compression. The EREC has been designed to exhibit graceful degradation with worsening channel conditions. The EREC is applicable to many problems and is particularly effective when the more important information is transmitted near the start of each variable-length block and is not dependent on following data. The EREC has been applied to both still image and video compression schemes, using the discrete cosine transform (DCT) and variable-length coding. The results have been compared to schemes using synchronization code words, and a large improvement in performance for noisy channels has been observed.

We examine how image coding schemes can be designed for noisy channels such as radio links, cordless telephones or mobile communications. We start with a JPEG based scheme and discuss its performance over channels with random and burst errors. We propose a number of modifications to the JPEG scheme including: avoiding DPCM coding of the DC coefficients; using a highly protected short header, the error resilient entropy code (EREC), error concealment and a wavelet transform. We show that these methods can improve the error resilience, without any significant increase of bit rate or loss of error free picture quality.>