Whole slide images (WSIs) are fundamental components of modern pathology, aiding pathologists in diagnosing diseases such as cancer. However, artefacts such as blurry regions, folded tissue, or uneven staining, often introduced during biopsy or slide preparation, can affect the accuracy and efficiency of the diagnostic process. Detecting these artefacts is critical to ensure delivering acceptable image quality, guiding pathologists toward diagnostically relevant regions, and improving reliability. In recent years, deep learning has increasingly complemented traditional quality control procedures by enabling rapid detection and localization of such artefacts. This progress, however, depends on the availability of high-quality datasets for training, validation, and testing of the detection models. To support this need, we introduce a comprehensive benchmark dataset for artefact detection obtained from two separate sources, consisting of selected WSIs from The Cancer Genome Atlas (TCGA), as well as WSIs provided by Universitair Ziekenhuis Brussel (UZB). The collected WSIs were subdivided into smaller tiles and annotated by artefact type, making them well-suited for deep learning model development. We hope this dataset serves as a valuable foundation for researchers developing tools to enhance WSI quality and diagnostic accuracy. The dataset is publicly available at: https://gitlab.com/etrovub/interfere/pathodataset25.

The Joint Photographic Experts Group (JPEG) is currently addressing the challenges in assessing plenoptic image quality by developing new standards for both subjective and objective assessment. This process entails revisiting existing recommendations and establishing a visual quality assessment (QA) framework that considers the unique aspects of plenoptic data. The focus is currently on the light field modality, with efforts to gather expert contributions and develop tools that cater both subjective and objective QA, ensuring that the evolving requirements of plenoptic imaging QA are met. This paper presents the JPEG Pleno subjective test tool, a pivotal element in JPEG’s standardization endeavor, designed to facilitate a range of subjective QA experiments that steer the decisions during a standardization process.

Deep features of convolutional neural networks (CNNs), designed for high-level computer vision tasks such as object detection, have been shown to be also effective for image quality assessment (IQA). This motivates further investigations to understand why CNNs are good IQA estimators and, in a broader sense, whether a correspondence exists between CNN's processing stages and the hierarchical mechanism of the human visual system (HVS). In this paper, we stimulate CNNs with a new family of maximally-regular textures to investigate if higher areas of the visual cortex can be approximated within the CNN layers. The results show interesting correspondence between CNNs and the visual cortex, suggesting that these frameworks might be able to serve as suitable baseline candidates for developing a new generation of quality metrics that better replicate the complex stages of the HVS. Developing such IQA models needs in-depth research into the mechanism of the HVS and designing CNNs with better quality-aware feature encoding. As an initial step toward this goal, we established new criteria for improving the performance of pre-trained networks in quality assessment applications by leveraging texture sensitivity. The outcomes illustrate that our feature map weighting and neuron selection criteria could improve the IQA task.

Explainable artificial intelligence (XAI) has gained considerable attention in recent years as it aims to help humans better understand machine learning decisions, making complex black-box systems more trustworthy. Visual explanation algorithms have been designed to generate heatmaps highlighting image regions that a deep neural network focuses on to make decisions. While convolutional neural network (CNN) models typically follow similar processing operations for feature encoding, the emergence of vision transformer (ViT) has introduced a new approach to machine vision decision-making. Therefore, an important question is which architecture provides more human-understandable explanations. This paper examines the explain-ability of deep architectures, including CNN and ViT models under different vision tasks. To this end, we first performed a subjective experiment asking humans to highlight the key visual features in images that helped them to make decisions in two different vision tasks. Next, using the human-annotated images, ground-truth heatmaps were generated that were compared against heatmaps generated by explanation methods for the deep architectures. Moreover, perturbation tests were performed for objective evaluation of the deep models' explanation heatmaps. According to the results, the explanations generated from ViT are deemed more trustworthy than those produced by other CNNs, and as the features of the input image are more dispersed, the advantage of the model becomes more evident.

Opinion-unaware no-reference (OU-NR) methods for image quality assessment (IQA) are of great interest since they can predict visual quality independent of a reference image and knowledge of human quality opinions. Models of image naturalness trained on a corpus of pristine images have shown potential for developing OU-NR methods. However, the extracted features may not match the preferences of the human visual system (HVS). This paper aims to utilize the features of convolutional neural networks to achieve a richer representation of the naturalness space. In addition, the IQA processing steps from training to quality measurement are revisited and the naturalness model is improved by incorporating HVSinspired criteria. Experimental results show the higher performance and generalizability of the naturalness model – constructed using HVS-aligned deep features – under different distortion types and image contents. The source code of the quality index is available at https://gitlab.com/saeedmp/dni.

At its 93rd meeting, JPEG received the responses to the Call for Proposals on JPEG Pleno holography. This activity will lead to the first standardized coding solution for digital holograms. This paper describes the quality evaluation process, existing out of a subjective evaluation study and an objective evaluation based on preselected quality metrics. It also includes a critical evaluation of the challenges encountered when performing quality evaluation of holographic content. Based on the evaluation of the response, the JPEG committee was able to select a very reliable coding solution that depicted a performance well beyond that of the selected anchors.

The Fast Fourier Transform (FFT) is an efficient algorithm widely used in digital signal processing to transform between the time domain and the frequency domain. For fixed-point VLSI implementations, dynamic range growth inevitably occurs at each stage of the FFT operation. However, current methods either waste bitwidth or consume excessive resources when dealing with the dynamic range growth issue. To address this issue, we propose an efficient scaling method called Scaling Information Propagation (SIP) to alleviate the problem of dynamic range growth, which makes full use of bitwidth with much less extra area consumed than the state-of-the-art solutions. In two consecutive transform operations, the SIP method extracts scaling information and makes scaling decisions in the former transform, then executes those in the latter one. We implement the FFT’s VLSI architecture in the orthogonal frequency division multiplexing (OFDM) and the holographic video compression (HVC) systems to verify the SIP method. Compared to the state-of-the-art, experimental results after VLSI synthesis show that our method achieves 9.38% energy reduction and 8.36% area savings when requiring 1.02 × 10-7bit error ratio (BER) of the OFDM system, and 33.47% energy reduction and 30.98% area savings when requiring 20dB signal-to-noise ratio (SNR) of the HVC system, respectively.

In preparation of the Call for Proposals on JPEG Pleno Holography, multiple exploration studies are ongoing to define the general procedure for performance assessment of proposed coding solutions. The performance of proposals will be compared against JPEG 2000 and H.265/HEVC intra coding, both serving as anchor codecs. In this paper, we report on the results of a dynamic subjective visual quality assessment procedure. A subset of 8 holograms, selected from the JPEG Pleno Database, is compressed both in hologram and object plane at three different bit rates. Chosen bit rates vary per hologram depending on their content characteristics. Then, for each hologram and its compressed versions, pseudo-video sequences are generated from the reconstructed views along a scan path that involves focus and viewing angle changes and that is hologram specific. A double stimulus simultaneous test, combined with a 5-level impairment-scale scoring protocol is deployed where videos created from the reference and the decoded holograms are visualized side by side on a professional 4K display. Results demonstrate that this test procedure requires deep scenes and sufficient scene complexity throughout the depth stack to allow for adequate stress testing of the codecs under test, particularly those solutions that adhere to compression in the object plane.

Objective quality assessment of digital holograms has proven to be a challenging task. While prediction of perceptual quality of the recorded 3D content from the holographic wavefield is an open problem; perceptual quality assessment from content after rendering, requires a time-consuming rendering step and a multitude of possible viewports. In this research, we use 96 Fourier holograms of the recently released HoloDB database to evaluate the performance of well-known and state-of-the-art image quality metrics on digital holograms. We compare the reference holograms with their distorted versions: (i) before rendering on the real and imaginary parts of the quantized complex-wavefield, (ii) after converting Fourier to Fresnel holograms, (iii) after rendering, on the quantized amplitude of the reconstructed data, and (iv) after subsequently removing speckle noise using a Wiener filter. For every experimental track, the quality metric predictions are compared to the Mean Opinion Scores (MOS) gathered on a 2D screen, light field display and a holographic display. Additionally, a statistical analysis of the results and a discussion on the performance of the metrics are presented. The tests demonstrate that while for each test track a few quality metrics present a highly correlated performance compared to the multiple sets of available MOS, none of them demonstrates a consistently high-performance across all four test-tracks.

Human visual system (HVS) strongly responds to motion information. In particular, neurons in middle temporal (MT) area of the brain are sensitive to certain velocities. Thus distortions appeared on moving stimuli, as visually salient information, are visually very important and can significantly affect the quality of experience (QoE). Judder is a visual artifact that manifests as non-smooth motion when tracking a moving object on a digital display. In particular, the effect of judder becomes more significant in the wide field of view (FoV) displays where objects can be observed in a longer trajectory and duration. Head mounted display (HMD) maximizes FoV by enabling full 360° immersive experience and there has been rapid growth in the number of omnidirectional videos for HMDs. However, the impact of judder on visual QoE of the omnidirectional video has never been studied. In this paper, we first established a database of omnidirectional video sequences to study human responses on the visual effect of judder when watching videos on wide FoV HMDs. Two subjective tests were conducted to understand the impact of judder on video sequences compressed at different bit rates (i.e. different picture quality levels). Next, based on the subjective results, we proposed a novel QoE model considering the joint effect of judder, visual masking and picture quality. Experimental results indicate the effectiveness of the proposed model for accurate quality prediction of the videos with judder.

Computer-Generated Holography (CGH) algorithms simulate numerical diffraction, being applied in particular for holographic display technology. Due to the wave-based nature of diffraction, CGH is highly computationally intensive, making it especially challenging for driving high-resolution displays in real-time. To this end, we propose a technique for efficiently calculating holograms of 3D line segments. We express the solutions analytically and devise an efficiently computable approximation suitable for massively parallel computing architectures. The algorithms are implemented on a GPU (with CUDA), and we obtain a 70-fold speedup over the reference point-wise algorithm with almost imperceptible quality loss. We report real-time frame rates for CGH of complex 3D line-drawn objects, and validate the algorithm in both a simulation environment as well as on a holographic display setup.

With the emergence of light field (LF) imaging technology, various challenging problems for LF visualization, compression, and transmission have to be addressed. Measuring perceptual quality is of utmost importance to assess the impact of an LF processing step on the visual experience of the finally rendered content to the end-user. In particular, objective quality assessment (QA) plays a key role in the quality optimization of LF imaging systems. In this paper, we conducted a comprehensive experiment to evaluate the performance of different objective QA methods for LF application. To this end, we selected a total number of 250 LFs (more than 48000 perspective views) from three public data sets to evaluate 16 objective QA metrics. Moreover, the subjective scores from three test data sets were aligned to produce an integrated data set for quality evaluation. The performance results across the different data sets aid to choose the most reliable metrics that are consistently performing well under various distortion and content characteristic conditions.

Depth Image-Based Rendering (DIBR) is the key technique in many multi-view 3D applications to synthesize virtual views using texture and depth information. However, DIBR induces distortions that disturb the visual quality of experience; therefore, image quality assessment (IQA) methods are essential to evaluate the quality of the synthesized views. The characteristics of the DIBR-related distortions are different from those of the traditional video coding distortions and conventional objective IQA methods often fail to provide accurate quality predictions for synthesized views. In this letter, we proposed a new Full Reference (FR) objective metric for evaluation of DIBR synthesized views. We used a feature matching method at feature (key) points of reference and synthesized images to quantify the local differences. Moreover, global quality loss is computed in shift-compensated views by measuring the gradient difference in image superpixels. Performance evaluation on the three public data sets shows the effectiveness of the proposed model. A software release of the proposed method is available at https://gitlab.com/etro/ssdi.

High dynamic range (HDR) imaging enables capturing a wide range of luminance levels existing in real-world scenes. While HDR capturing devices become widespread in the market, the display technology is yet limited in representing full luminance ranges and standard low dynamic range (LDR) displays are currently more prevalent. To visualize the HDR content on traditional displays, tone mapping (TM) operators are introduced that convert HDR content into LDR. The dynamic range compression and different processing steps during TM can lead to loss of scene details, as well as luminance and chrominance changes. Such signal deviations will affect image naturalness and consequently disturb the visual quality of experience. Therefore, research into objective methods for quality evaluation of tone-mapped images has received attention in recent years. In this paper, we proposed a completely blind image quality evaluator for tone-mapped images based on a multi-attribute feature extraction scheme. Due to the diversity of TM distortions, various image characteristics are taken into account to develop an effective metric. The features are designed by considering spectral and spatial entropy, detection probability of visual information, image exposure, sharpness, and color properties. The quality-relevant features are then fed into a machine-learning regression framework to pool a quality score. The validation tests on two benchmark datasets reveal the superior performance of the proposed approach compared to the competing metrics.

Compression of digital holograms is a major challenge that needs to be resolved to enable the efficient storage, transmission and rendering of macroscopic holographic signals. In this work, we propose to deploy the wave atom transform that has been utilized before for interferometric modalities such as acoustic and seismic signals. This non-adaptive multiresolution transform has good space-frequency localization and its orthonormal basis is suitable for sparsifying holographic signals. By replacing the CDF 9/7 wavelet transform stage in a JPEG 2000 codec with the proposed wave atom transform, we did assess its suitability for coding complex amplitude wavefronts. Experimental results demonstrate improved rate-distortion performance with respect to JPEG 2000 and H.265/HEVC for a set of computer-generated, diffuse, macroscopic holograms.

Digital holograms fully encode the wavefield of light, thereby having many applications both for 3D object measurements as well as for display purposes, accounting for all human visual cues. Because the statistics and properties of holographic signals differ considerably from natural imagery such as photographs, conventional coding solutions will be sub-optimal. In this paper, we propose the integer Fresnel transform, which is - to our knowledge - the first lossless transform tailored for hologram coding. By combining the proposed transform with JPEG 2000, we report bit-rate savings from 0.12 up to 2.83 bits per channel on a collection of 8 digital holograms obtained from 3 different databases.

Holography is considered to be the ultimate display technology since it can account for all human visual cues such as stereopsis and eye focusing. Aside from hardware constraints for building holographic displays, there are still many research challenges regarding holographic signal processing that need to be tackled. In this overview, we delineate the steps needed to realize an end-to-end chain from digital content acquisition to display, involving the efficient generation, representation, coding and quality assessment of digital holograms. We discuss the current state-of-the-art and what hurdles remain to be taken to pave the way towards realistic visualization of dynamic holographic content.

This work builds upon the kernel regression framework for solving the general image processing problem of denoising, deblurring and interpolating from scattered image samples. A competitive expectation-maximization method estimates globally all parameters of a generative image model, accounting for missing samples. One 2D footprint kernel and a local linear regression plane are estimated per data sample. Kernels can shift and their prior probabilities are estimated as well, unlike in nonparametric models. Missing data yields an underdetermined problem that is regularized by smoothing the marginal mixture density. At each iteration, a balloon estimator computes numerically the spatial “territory” associated to each data samples. Results of these numerical diffusion operations are used to convolve adaptively each kernel in the forward model. Finally, the complete image is reconstructed by smoothing regression for combining conditional means of local linear regressors. Experiments apply this iterative Bayesian technique in image restoration.

High Dynamic Range (HDR) imaging captures a wide range of luminance existing in real-world scenes. Due to large luminance levels and higher brightness of HDR displays, artefacts can be more noticeable to the Human Visual System (HVS). In a first attempt to experimentally quantify those noticeable levels for HDR images, we pioneered in conducting an exhaustive and comprehensive Just Noticeable Difference (JND) subjective experiment of which the outcome is presented in this paper. Six distortions including JPEG, JPEG2000, noise, blur, contrast change, and quantization artefacts have been considered in the test. The distortions were applied to 10 HDR images in 100 distortion levels resulting a database of 6000 HDR test images. The subjects were asked to find the image JND location on each set of 100 images they had the freedom to explore. The effect of content features on the noticeable threshold selection is investigated per distortion type. Our results in some cases show a significant correlation between content features and JNDs. We are hoping that our results can contribute to further exploitation of a precise HVS model for HDR quality assessment and optimization of the coding and bit allocation in HDR compression.

Fingerprinting codes based on the marking assumption provide reliable security against collusion attacks, but lack resilience against channel errors such as symbol erasures or other forms of distortions introduced by transmission over noisy or insecure communication channels. This additional source of distortion errors can be addressed by relaxing the marking assumption. In this paper we examine the restrictions and limitations on the code construction in terms of accusation errors, alphabet size, and the distortion errors under this weaker form of the marking assumption as formulated by Guth and Pfitzmann, also deriving a theoretical minimum lower bound on the code length of q-ary fingerprinting codes. We provide the formulas for applying the already existing binary Tardos code with symmetric accusation function under the weaker form of the marking assumption along with numerical results proving the tightness of the derived bounds. We show that the code is still of asymptotically minimum code length under the relaxation of the marking assumption.

An increasing number of image processing applications require an automated quality prediction of the visual content as perceived by humans. Since, sparse coding is suggested to be an underlying strategy of the brain's neural system, it would be logical to assume that specific tasks like quality assessment also attempt to adhere to this strategy. However, existing perceptual quality predictors, often mimicking the different stages of the human visual system and deploying machine learning strategies, such as neural networks, rarely integrate the concept of sparse coding in their design. In this paper, we first investigate the validity of such assumption by performing an empirical analysis on the relation between the structural information of the scene-captured via sparseness significance-and perceptual quality. Subsequently, we propose a new approach to integrate the significance of sparse coding features in the future imagequality measure (IQM) designs. We utilize the Fourier transform as a case study, which leads to a new IQM called sparseness significance ranking measure (SSRM). This measure essentially deploys a Fourier basis for sparse coding, a ranking mechanism based upon the amplitudes of the sparse coefficients and subsequently a complex correlation metric that assesses the correspondence between the ranked coefficient amplitude profiles of the reference and the distorted image. Moreover, we introduce a new methodology, namely separation ratio analysis, to assess the prediction quality of individual features or quality predictors given a target perceptual quality. The quality predictions by the proposed SSRM show excellent compatibility with perceptual quality scores. A set of routine benchmarking experiments utilizing the LIVE and CSIQ, IVC and TID2008 databases indicates a highly competitive performance with state of the art IQMs. Moreover, it delivers this performance at a low computational cost.

During acquisition, processing, compression and transmission, images may be corrupted by multiple distortions such as blur, noise or compression artefacts. However, current image quality assessment (IQA) methods are often designed for images degraded by a single distortion type. This paper proposes a reduced-reference (RR) IQA method to predict the quality of multi-distorted images. The method is based on feature extraction from the reference and the distorted images. Based on internal generative mechanism (IGM) theory, the images are decomposed first into their predicted and disorderly portions. Next, several features are captured from each portion and feature differences are computed between the reference and distorted images. Finally, support vector regression (SVR) is adopted to obtain a quality score. The results on public multiply-distorted image databases, namely MDID2015 and MLIVE, show that the proposed method delivers higher accuracy than several image quality metrics.

Polygonal meshes are popular three-dimensional virtual representations employed in a wide range of applications. Users have very high expectations with respect to the accuracy of these virtual representations, fueling a steady increase in the processing power and performance of graphics processing hardware. This accuracy is closely related to how detailed the virtual representations are. The more detailed these representations become, the higher the amount of data that will need to be displayed, stored, or transmitted. Efficient compression techniques are of critical importance in this context. State-ofthe-art compression performance of semi-regular mesh coding systems has been achieved through the use of subdivision-based wavelet coding techniques. However, the vast majority of these codecs are optimized with respect to the L2distortion metric, i.e., the average error. This makes them unsuitable for applications where each input signal sample has a certain significance. To alleviate this problem, we propose to optimize the mesh codec with respect to the L∞metric, which allows for the control of the local reconstruction error. This paper proposes novel data-dependent formulations for the L∞distortion. The proposed L∞estimators are incorporated in a state-of-the-art wavelet-based semi-regular mesh codec. The resulting coding system offers scalability in L∞sense. The experiments demonstrate the advantages of L∞coding in providing a tight control on the local reconstruction error. Furthermore, the proposed data-dependent L∞approaches significantly improve estimation accuracy, reducing the classical low-rate gap between the estimated and actual L∞distortion observed for previous L∞estimators.

View synthesis using depth image-based rendering generates virtual viewpoints of a 3D scene based on texture and depth information from a set of available cameras. One of the core components in view synthesis is image inpainting which performs the reconstruction of areas that were occluded in the available cameras but are visible from the virtual viewpoint. Inpainting methods based on Markov random fields (MRFs) have been shown to be very effective in inpainting large areas in images. In this paper, we propose a novel MRF-based in-painting method for multiview video. The proposed method steers the MRF optimization towards completion from background to foreground and exploits the available depth information in order to avoid bleeding artifacts. The proposed approach allows for efficiently filling-in large disocclusion areas and greatly accelerates execution compared to traditional MRF-based inpainting techniques. The experimental results show that view synthesis based on the proposed inpainting method systematically improves performance over the state-of-the-art in multiview view synthesis. Average PSNR gains up to 1.88 dB compared to the MPEG View Synthesis Reference software were observed.

This paper reports results of subjective and objective quality assessments of responses to a grand challenge on light field image compression. The goal of the challenge was to collect and evaluate new compression algorithms for light field images. In total seven proposals were received, out of which five were accepted for further evaluations. For objective evaluations, conventional metrics were used, whereas the double stimulus continuous quality scale method was selected to perform subjective assessments. Results show competitive performance among submitted proposals. However, in low bitrates, one proposal outperforms the others.

Advanced sparse sampling acquisition systems capture only scattered information from the continuous image domain. Unfortunately, conventional image encoders are not yet able to properly compress arbitrarily subsampled image data. This work introduces a system leveraging the JPEG 2000 image compression framework by enabling scalable compression of the selected image samples. Using a complete dictionary of CDF 9/7 wavelets, a minimum l1-norm compressed sensing solution is recovered which can be fed directly into the encoder, producing a bitstream that can be decoded with existing JPEG 2000-compliant implementations. Experiments on standard images with quasi-random subsampling demonstrate that the proposed system outperforms regular JPEG 2000 compression of stacked sample images and quad-tree based compression for point-clouds. We also demonstrate the robustness of the technique for images that infringe the sparsity prior of compressed sensing.

Compressive sensing is a mathematical framework, which seeks to capture the information of an object using as few measurements as possible. Recently, it has been applied to holography, where the most frequently used reconstruction method is l1-norm minimization with the Haar wavelet as the sparsifying operator. In this work, we promote the CDF 9/7 wavelet as the sparsifying operator. We demonstrate that the CDF 9/7 wavelet performs better than the Haar wavelet.

We present the implementation of a volumetric JPEG 2000 codec as a real-world use case of software acceleration with GPUs and multi-core CPUs. We present a generic methodology to accelerate existing code written in C with OpenCL. Furthermore, we account for the volumetric nature of the processed data and formulate associated optimization guidelines. The resulting software can exploit different accelerator types - GPUs and multi-core CPUs - and delivers a decent speedup on a variety of hardware platforms for a relatively small effort.

The amount of image data generated each day in health care is ever increasing, especially in combination with the improved scanning resolutions and the importance of volumetric image data sets. Handling these images raises the requirement for efficient compression, archival and transmission techniques. Currently, JPEG 2000׳s core coding system, defined in Part 1, is the default choice for medical images as it is the DICOM-supported compression technique offering the best available performance for this type of data. Yet, JPEG 2000 provides many options that allow for further improving compression performance for which DICOM offers no guidelines. Moreover, over the last years, various studies seem to indicate that performance improvements in wavelet-based image coding are possible when employing directional transforms. In this paper, we thoroughly investigate techniques allowing for improving the performance of JPEG 2000 for volumetric medical image compression. For this purpose, we make use of a newly developed generic codec framework that supports JPEG 2000 with its volumetric extension (JP3D), various directional wavelet transforms as well as a generic intra-band prediction mode. A thorough objective investigation of the performance-complexity trade-offs offered by these techniques on medical data is carried out. Moreover, we provide a comparison of the presented techniques to H.265/MPEG-H HEVC, which is currently the most state-of-the-art video codec available. Additionally, we present results of a first time study on the subjective visual performance when using the aforementioned techniques. This enables us to provide a set of guidelines and settings on how to optimally compress medical volumetric images at an acceptable complexity level.

When compared to conventional 2-D video, multiview video can significantly enhance the visual 3-D experience in 3-D applications by offering horizontal parallax. However, when processing images originating from different views, it is common that the colors between the different cameras are not well- calibrated . To solve this problem, a novel energy function -based color correction method for multiview camera setups is proposed to enforce that colors are as close as possible to those in the reference image but also that the overall structural information is well-preserved. The proposed system introduces a spatio-temporal correspondence matching method to ensure that each pixel in the input image gets bijectively mapped to a reference pixel. By combining this mapping with the original structural information, we construct a global optimization algorithm in a Laplacian matrix formulation and solve it using a sparse matrix solver. We further introduce a novel forward-reverse objective evaluation model to overcome the problem of lack of ground truth in this field. The visual comparisons are shown to outperform state-of-the-art multiview color correction methods, while the objective evaluation reports PSNR gains of up to 1.34 dB and SSIM gains of up to 3.2%, respectively.

To automate quality monitoring of multimedia applications, objective quality measures for images and video content need to be designed. Objective quality measures that model the Human Visual System (HVS) have a disappointing performance, because the HVS is not sufficiently understood. Integrating machine learning (ML) techniques may increase the performance. Unfortunately, traditional ML is difficult to interpret. To this end, we developed the Locally Adaptive Fusion (LAF), for more flexible and reliable quality predictions. This manuscript proposes six interactive programs and a website that demonstrate the effectiveness of LAF, which complement the technical focus of the corresponding journal paper.

Image privacy and security are receiving increasingly more attention these days though their impact is still underestimated. Every day billions of digital pictures are generated and distributed via social media, news sites and photo sharing applications. These images tend to carry quite some information that can be regarded to be private for the photographer and/or the pictured individuals. Furthermore, this information is not solely constrained to the visible information carried by the picture. For instance, the sensor noise that is embedded in the picture provides information related to the camera used to take the photo. Besides the picture essence also metadata is signaled in the image file containers, providing geospatial information, camera settings, photographer related information, etc. Hence, it is evident to understand that to protect the privacy of the persons that can be associated with the picture, the necessary security precautions need to be taken. Since many pictures are represented in JPEG image formats, it is evident that the JPEG committee is also investigating how it can integrate better support for privacy and security requirements in its suite of standards. In this presentation, an overview will be given of standardized solutions, where we will particularly focus on JPEG 2000 JPSEC framework, but will also discuss how the committee plans to further improve the support for these requirements in the context of the legacy JPEG standard.

Objective measures to automatically predict the perceptual quality of images or videos can reduce the time and cost requirements of end-to-end quality monitoring. For reliable quality predictions, these objective quality measures need to respond consistently with the behavior of the human visual system (HVS). In practice, many important HVS mechanisms are too complex to be modeled directly. Instead, they can be mimicked by machine learning systems, trained on subjective quality assessment databases, and applied on predefined objective quality measures for specific content or distortion classes. On the downside, machine learning systems are often difficult to interpret and may even contradict the input objective quality measures, leading to unreliable quality predictions. To address this problem, we developed an interpretable machine learning system for objective quality assessment, namely the locally adaptive fusion (LAF). This paper describes the LAF system and compares its performance with traditional machine learning. As it turns out, the LAF system is more consistent with the input measures and can better handle heteroscedastic training data.

Wyner-Ziv video coding constitutes an alluring paradigm for visual sensor networks, offering efficient video compression with low complexity encoding characteristics. This work presents a novel hash-driven Wyner-Ziv video coding architecture for visual sensors, implementing the principles of successively refined Wyner-Ziv coding. To this end, so-called side-information refinement levels are constructed for a number of grouped frequency bands of the discrete cosine transform. The proposed codec creates side-information by means of an original overlapped block motion estimation and pixel-based multihypothesis prediction technique, specifically built around the pursued refinement strategy. The quality of the side-information generated at every refinement level is successively improved, leading to gradually enhanced Wyner-Ziv coding performance. Additionally, this work explores several temporal prediction structures, including a new hierarchical unidirectional prediction structure, providing both temporal scalability and low delay coding. Experimental results include a thorough evaluation of our novel Wyner-Ziv codec, assessing the impact of the proposed successive refinement scheme and the supported temporal prediction structures for a wide range of hash configurations and group of pictures sizes. The results report significant compression gains with respect to benchmark systems in Wyner-Ziv video coding (e.g., up to 42.03% over DISCOVER) as well as versus alternative state-of-the-art schemes refining the side-information.

This paper presents a novel still image compression scheme that extends the traditional JPEG standard with lossy-to-lossless compression support. The system follows a two-layer design approach which allows for backward compatibility with the conventional JPEG standard for the base layer and provides lossless compression when decoding the enhancement layer. The system employs several coding tools, including quadtree coding, spatial domain prediction, reversible discrete cosine transforms, and context-based arithmetic coding to efficiently encode losslessly the enhancement layer. Performance evaluations using a standard JPEG set of images show that the proposed system yields similar lossless compression performance to the state-of-the-art single-layer JPEG-LS standard while providing quality scalability and a JPEG-compatible base layer at the same time.

Currently, the use of the 2D wavelet transform in texture compression for real-time texture mapping on the GPU is limited. The main cause of this is the lack of real-time texture filtering implementations which do not require specialized hardware. This work proposes a novel system to perform 2D wavelet reconstruction and bilinear texture filtering using a high performance GPU shader. The system is able to generate a performant GLSL shader for arbitrary wavelet filter configurations. This goes beyond earlier works in the literature proposing Haar wavelet and Discrete Cosine Transform (DCT) implementations on the GPU. We analyse the shader performance and run-time complexity for several wavelet filters. The experimental results show that filters longer than Haar are deployable on the GPU while maintaining accurate texture filtering and real-time performance.

This paper presents a novel two-layer coding framework targeting visually lossless compression of screen content video. The proposed framework employs the conventional HEVC standard for the base-layer. For the enhancement layer, a hybrid of spatial and temporal block-prediction mechanism is introduced to guarantee a small energy of the error-residual. Spatial prediction is generally chosen for dynamic areas, while temporal predictions yield better prediction for static areas in a video frame. The prediction residual is quantized based on whether a given block is static or dynamic. Run-length coding, Golomb based binarization and context-based arithmetic coding are employed to efficiently code the quantized residual and form the enhancement-layer. Performance evaluations using 4:4:4 screen content sequences show that, for visually lossless video quality, the proposed system significantly saves the bit-rate compared to the two-layer lossless HEVC framework.

We propose a novel image authentication system by combining perceptual hashing and robust watermarking. An image is divided into blocks. Each block is represented by a compact hash value. The hash value is embedded in the block. The authenticity of the image can be verified by re-computing hash values and comparing them with the ones extracted from the image. The system can tolerate a wide range of incidental distortion, and locate tampered areas as small as 1/64 of an image. In order to have minimal interference, we design both the hash and the watermark algorithms in the wavelet domain. The hash is formed by the sign bits of wavelet coefficients. The lattice-based QIM watermarking algorithm ensures a high payload while maintaining the image quality. Extensive experiments confirm the good performance of the proposal, and show that our proposal significantly outperforms a state-of-the-art algorithm.

In the context of low-cost video encoding, distributed video coding (DVC) has recently emerged as a potential candidate for uplink-oriented applications. This paper builds on a concept of correlation channel (CC) modeling, which expresses the correlation noise as being statistically dependent on the side information (SI). Compared with classical side-information-independent (SII) noise modeling adopted in current DVC solutions, it is theoretically proven that side-information-dependent (SID) modeling improves the Wyner-Ziv coding performance. Anchored in this finding, this paper proposes a novel algorithm for online estimation of the SID CC parameters based on already decoded information. The proposed algorithm enables bit-plane-by-bit-plane successive refinement of the channel estimation leading to progressively improved accuracy. Additionally, the proposed algorithm is included in a novel DVC architecture that employs a competitive hash-based motion estimation technique to generate high-quality SI at the decoder. Experimental results corroborate our theoretical gains and validate the accuracy of the channel estimation algorithm. The performance assessment of the proposed architecture shows remarkable and consistent coding gains over a germane group of state-of-the-art distributed and standard video codecs, even under strenuous conditions, i.e., large groups of pictures and highly irregular motion content.

Triggered by the challenging prerequisites of wireless capsule endoscopic video technology, this paper presents a novel distributed video coding (DVC) scheme, which employs an original hash-based side-information creation method at the decoder. In contrast to existing DVC schemes, the proposed codec generates high quality side-information at the decoder, even under the strenuous motion conditions encountered in endoscopic video. Performance evaluation using broad endoscopic video material shows that the proposed approach brings notable and consistent compression gains over various state-of-the-art video codecs at the additional benefit of vastly reduced encoding complexity.

This paper presents a novel image adaptive data hiding system using properties of the discrete wavelet transform and which is ready to use in combination with JPEG 2000. Image adaptive watermarking schemes determine the embedding samples and strength from the image statistics. We propose to use the energy of wavelet coefficients at high frequencies to measure the amount of distortion that can be tolerated by a lower frequency coefficient. The watermark decoder in image adaptive data hiding needs to estimate the same parameters used for encoding from a modified source and hence is vulnerable to desynchronization. We present a novel way to resolve these synchronization issues by employing specialized insertion, deletion and substitution codes. Given the low complexity and reduced perceptual impact of the embedding technique, it is suitable for inserting camera and/or projector information to facilitate image forensics.

In this paper we present a remeshing algorithm which drastically reduces the aliasing artifacts inherent in regularly-sampled remeshed objects. Starting from a semi-regular mesh, the proposed algorithm reduces the remeshing error and avoids aliasing by displacing vertices such that most samples of the original mesh are present in the remeshed model as well. Computational efficiency is provided by using a search-tree, which efficiently gathers vertices near a given point in a 3D space. Compared to the state-of-the-art semi-regular remesher, the proposed remesher drastically improves the visual quality of the high-frequency regions in remeshed objects. Additionally, the proposed algorithm yields a lower remeshing error, which is reflected by a significantly increased PSNR upper-bound in wavelet-based compression of such meshes.

Our recently proposed wavelet-based L-infinite-constrained coding approach for meshes ensures that the maximum error between the vertex positions in the original and decoded meshes is guaranteed to be lower than a given upper bound. Instantiations of both L-2 and L-infinite coding approaches are demonstrated for MESHGRID, which is a scalable 3D object encoding system, part of MPEG-4 AFX. In this survey paper, we compare the novel L-infinite distortion estimator against the L-2 distortion estimator which is typically employed in 3D mesh coding systems. In addition, we show that, under certain conditions, the L-infinite estimator can be exploited to approximate the Hausdorff distance in real-time implementations.

This paper proposes novel scalable mesh coding designs exploiting the intraband or composite statistical dependencies between the wavelet coefficients. A Laplacian mixture model is proposed to approximate the distribution of the wavelet coefficients. This model proves to be more accurate when compared to commonly employed single Laplacian or generalized Gaussian distribution models. Using the mixture model, we determine theoretically the optimal embedded quantizers to be used in scalable wavelet-based coding of semiregular meshes. In this sense, it is shown that the commonly employed successive approximation quantization is an acceptable, but in general, not an optimal solution. Novel scalable intraband and composite mesh coding systems are proposed, following an information-theoretic analysis of the statistical dependencies between the coefficients. The wavelet subbands are independently encoded using octree-based coding techniques. Furthermore, context-based entropy coding employing either intraband or composite models is applied. The proposed codecs provide both resolution and quality scalability. This lies in contrast to the state-of-the-art interband zerotree-based semiregular mesh coding technique, which supports only quality scalability. Additionally, the experimental results show that, on average, the proposed codecs outperform the interband state-of-the-art for both normal and nonnormal meshes. Finally, compared with a zerotree coding system, the proposed coding schemes are better suited for software/hardware parallelism, due to the independent processing of wavelet subbands.

The paper investigates the novel concept of local-error control in mesh geometry encoding. In contrast to traditional mesh-coding systems that use the mean-square error as target distortion metric, this paper proposes a new L-infinite mesh-coding approach, for which the target distortion metric is the L-infinite distortion. In this context, a novel wavelet-based L-infinite-constrained coding approach for meshes is proposed, which ensures that the maximum error between the vertex positions in the original and decoded meshes is lower than a given upper bound. Furthermore, the proposed system achieves scalability in L-infinite sense, that is, any decoding of the input stream will correspond to a perfectly predictable L-infinite distortion upper bound. An instantiation of the proposed L-infinite-coding approach is demonstrated for MESHGRID, which is a scalable 3D object encoding system, part of MPEG-4 AFX. In this context, the advantages of scalable L-infinite coding over L-2-oriented coding are experimentally demonstrated. One concludes that the proposed L-infinite mesh-coding approach guarantees an upper bound on the local error in the decoded mesh, it enables a fast real-time implementation of the rate allocation, and it preserves all the scalability features and animation capabilities of the employed scalable mesh codec.

The paper thoroughly validates the proposed SID model on a broad dataset, showing significant accuracy improvements over SII models. Additionally, it is theoretically demonstrated that DVC systems that make SII assumptions suffer a performance penalty depending entirely on the correlation statistics of the video data.

Current models in Wyner-Ziv video coding consider the temporal correlation noise to be side-information independent (SII). This paper goes beyond this assumption and proposes a novel model, of which the parameters are side-information dependent (SID). The proposed model is experimentally validated showing remarkable accuracy improvement over the conventional SII model. Moreover, a novel SID technique for the accurate estimation of the correlation channel in video is introduced. The proposed technique enables the design of a novel pixel-domain Wyner-Ziv video coding system operating without a feedback channel. Preliminary experimental results show that the proposed codec achieves superior performance compared to the state-of-the-art in pixel-domain Wyner-Ziv coding.

In this paper, a novel wavelet-based composite mesh coding scheme is presented. In contrast to the state-of-the-art scalable interband mesh codec, the proposed codec relies on composite dependency models that capture both the interband and intraband statistical dependencies between wavelet coefficients. Additionally, each wavelet subband is processed independently, which allows for parallelized processing and for a progressive reconstruction of each mesh resolution level. Compared to the state-of-the-art, the proposed codec yields for almost all rate points superior compression performance in L2-sense. Furthermore, the generated bitstreams are near-optimal in rate-distortion sense, which eliminates the need of a post-compression rate-distortion optimization technique.

It has been recently demonstrated that, in distributed video coding (DVC) side-information dependent modeling of the correlation channel brings significant performance gains over side-information independent assumptions. In this letter, we present a novel technique enabling advanced side-information dependent estimation of the correlation channel at the decoder starting from a very coarse knowledge of it. The proposed technique triggers the design of a spatial-domain DVC codec which enables the suppression of the feedback channel. Experimental results show that the proposed codec achieves similar performance compared to the state-of-the-art in transform-domain Wyner-Ziv video coding while still operating at a fraction of its encoding complexity.

Integrating video coding and denoising is a novel processing paradigm, bringing mutual benefits to both video processing tools. In this paper, we propose a novel video denoising approach of which the main idea is reusing motion estimation resources from the video coding module for video denoising. In most cases, the motion fields produced by real-time video codecs cannot be directly employed in video denoising, since they, as opposed to noise filters, tolerate errors in the motion field. In order to solve this problem, we propose a novel motion-field filtering step that refines the accuracy of the motion estimates to a degree that is required for denoising. Additionally, a novel temporal filter is proposed that is robust against errors in the estimated motion field. Numerical results demonstrate that the proposed denoising scheme is of low-complexity and compares favorably to the state-of-the-art video denoising methods.

This paper investigates the novel concept of local error control in arbitrary mesh encoding, and proposes a new L-infinite mesh coding approach implementing this concept. In contrast to traditional mesh coding systems that use the mean-square error as distortion measure, the proposed approach employs the L-infinite distortion as target distortion metric. In this context, a novel wavelet-based L-infinite-constrained coding approach for meshes is proposed, which ensures that the maximum local error between the original and decoded meshes is lower than a given upper-bound. Additionally, the proposed system achieves scalability in L-infinite sense, that is, the L-infinite distortion upper-bound can be accurately estimated when decoding any layer from the input stream. Moreover, a distortion estimation approach is proposed, expressing the L-infinite distortion in the spatial domain as a statistical estimate of quantization errors produced in the wavelet domain. An instantiation of the proposed L-infinite coding approach is demonstrated for MESHGRID, which is a scalable 3D object coding system, part of MPEG-4 AFX. The proposed L-infinite coding approach guarantees that the maximum error is upper-bounded, it enables a fast real-time implementation of the rate-allocation, and it preserves all the scalability features and animation capabilities of the employed scalable mesh codec.

A novel coding approach, applying open-loop coding principles in predictive coding systems is proposed in this paper. The proposed approach is instantiated with an intra-frame video codec employing the transform and spatial prediction modes from H.264. Additionally, a novel rate-distortion model for open-loop predictive coding is proposed and experimentally validated. Optimally allocating rate based on the proposed model provides significant gains in comparison to a straightforward rate allocation not accounting for drift. Furthermore, the proposed open-loop predictive codec provides gains of up to 2.3 dB in comparison to an equivalent closed-loop intra-frame video codec employing the transform, prediction modes and rate-allocation from H.264. This indicates that, with appropriate drift compensation, open-loop predictive coding offers the possibility for further improving the compression performance in predictive coding systems.

This paper proposes a novel joint source-channel coding (JSCC) methodology which minimizes the end-to-end distortion for the transmission over packet loss channels of scalable video encoded using SVC, the scalable extension of H.264/AVC. The proposed JSCC approach performs channel protection using low-density parity-check codes and relies on Lagrangian-based optimization techniques to derive the appropriate protection levels for each layer produced by the scalable source codec. Our JSCC approach for SVC can support spatial, temporal and quality scalability and can provide an optimized channel protection in any scalable setting. Experiments show that our JSCC methodology yields competitive results against state-of-the-art Lagrangian-based JSCC algorithms. Compared to the state-of-the-art, our approach significantly reduces the number of computations needed to derive the rate-distortion hulls. Moreover, the proposed approach constructs convex rate-distortion hulls for each frame, irrespective of the target rate. This allows the pre-computation of the convex rate-distortion hulls for typical packet loss channels, such that the extraction of a near-optimal JSCC allocation can be achieved on-the-fly for any target rate or packet-loss rate. We conclude that the proposed JSCC methodology provides optimized resilience against transmission errors in scalable video streaming over variable-bandwidth error-prone channels.

This paper proposes a new approach for joint source and channel coding (JSCC) of meshes, simultaneously providing scalability and optimized resilience against transmission errors. An unequal error protection approach is followed, to cope with the different error-sensitivity levels characterizing the various resolution and quality layers produced by the input scalable source codec. The number of layers and the protection levels to be employed for each layer are determined by solving a joint source and channel coding problem. In this context, a novel fast algorithm for solving the optimization problem is conceived, enabling a real-time implementation of the JSCC rate-allocation. An instantiation of the proposed JSCC approach is demonstrated for MeshGrid, which is a scalable 3-D object representation method, part of MPEG-4 AFX. In this context, the L-inflnite distortion metric is employed, which is to our knowledge a unique feature in mesh coding. Numerical results show the superiority of the L-inflnite norm over the classical L-2 norm in a JSCC setting. One concludes that the proposed joint source and channel coding approach offers resilience against transmission errors, provides graceful degradation, enables a fast real-time implementation, and preserves all the scalability features and animation capabilities of the employed scalable mesh codec.

Distributed video coding is a coding paradigm that allows complexity to be shared between encoder and decoder. In this context, video coding systems have been developed with encoder complexities similar to H.263+ intra-coding, while obtaining compression performance comparable to H.263+ inter coding. The decoders in these systems typically employ motion-compensated frame interpolation or extrapolation to generate side-information. However, as motion complexity of the video sequence increases, such generators fail to provide reliable side information. This paper proposes a pixel-domain distributed video coding method, combining low-complexity encoder-side bitplane motion estimation with decoder-side motion-compensated frame interpolation. It is shown that such a system is more suitable for sequences with increased motion complexity, compared to codecs that employ motion estimation at the decoder only.

This paper proposes a novel segmentation-driven direction-adaptive discrete wavelet transform (SD DADWT), wherein the adaptation of the directional wavelet bases is performed on the segments describing the natural geometry of the image. First, a multi-resolution segmentation of the image is performed, obtained through an Edgmentation procedure. The optimum lifting directions are then selected for each segment and at each resolution. The proposed SD DADWT retains the inherent advantages offered by a multiresolution representation of the geometric features in the image, and in the same time provides a sparse image representation via DADWT. Preliminary experimental results obtained in a coding application show that the visual quality of the reconstructed image can be further improved by applying a geometrically-oriented transform on segments that approximate the natural borders in the image.

This paper proposes an optimized joint source-channel coding methodology with unequal error protection for the transmission of video encoded with the recently developed scalable extension of H.264/MPEG-4 AVC. The proposed methodology uses a simplified Viterbi-based search method which significantly outperforms the classical exhaustive search method in terms of computational complexity, leading to a practically applicable solution at the expense of a minimal loss of optimality. Experimental results show the effectiveness of our protection methodology and illustrate its capability to provide graceful degradation in the presence of channel mismatches.

In this paper, we propose a joint source-channel coding (JSCC) methodology which minimizes the end-to-end distortion for the transmission of H.264/MPEG-4 scalable video over packet loss channels. The proposed JSCC-approach employs low-density parity-check codes in order to provide channel protection and relies on Lagrangian-based optimization techniques to derive the appropriate protection levels for each layer produced by the scalable source codec. Experiments show that our JSCC methodology delivers competitive results to state-of-the-art Lagrangian-based algorithms. However, in contrast to the state-of-the-art, our approach significantly reduces the computational complexity. We conclude that the proposed JSCC methodology provides optimized resilience against transmission errors in scalable video streaming over variable-bandwidth error-prone channels.

Video source, carrier and client diversification have led the video coding community to develop scalable video codecs supporting efficient decoding at varying resolution, frame rate and quality. Scalable video has several advantages over a nonscalable approach, but a large scale deployment is far from trivial and a lot of open questions remain. To resolve these, we developed a complete video delivery chain for scalable wavelet-based video. This includes a video server, a negotiation framework, a video scaling infrastructure and two scalable video clients, one pure software client and one real-time, hardware accelerated client. This paper describes the complete chain and identifies and quantifies the impact of using scalable video in every link of this chain.

Unequal error protection (UEP) is the most practical way to achieve joint source-channel coding while keeping a clean separation between the source and channel coders. In this paper, a generic UEP allocation methodology is proposed. The methodology is based on a run-time algorithm, exploiting design-time models, and is applicable to bit-level transmission scenarios as well as packet-based transmission environments. It works with source-independent modeling and a linear complexity with respect to the number of substreams in the source coded stream and the number of protection levels available, which makes it an attractive solution for practical scenarios. We obtain a substantial complexity reduction compared to a classical Lagrangian optimization while keeping optimality in the allocation of the protection levels. Given a typical source and channel granularity, we obtain a factor 6 complexity reduction.

This paper reports an information-theoretic analysis of the inter-scale, inter-orientation and inter-location dependencies that exist between curvelet coefficients. We show that the marginal statistics of these coefficients can be accurately modeled using generalized Gaussian density functions. Though generally decorrelated, we find that curvelets exhibit unusually high dependencies in intra-band local micro-neighborhoods, of a magnitude not found for instance in classical wavelets. Furthermore, dependencies are subject to and decrease with increasing orientation and location differences. Finally, we conclude that intra-band coefficient dependencies are stronger than either their inter-scale or inter-direction counterparts.

The joint photographic experts group (JPEG) committee (ISO/IEC JTC1/SC29/WG1) is currently pursuing the standardization of a three-dimensional extension of the JPEG-2000 standard (Parts 1 and 2) to support the encoding of volumetric data sets. This extension, Part 10 - extensions for three-dimensional data (JP3D), will support functionalities like resolution scalability, quality scalability and region-of-interest coding, while exploiting the entropy in the additional third dimension to improve the rate-distortion performance. In this paper, we give an overview of the markets and application areas targeted by JP3D, the imposed requirements and the algorithm under study

Among the different classes of coding techniques proposed in literature, predictive schemes have proven their outstanding performance in near-lossless compression. However, these schemes are incapable of providing embedded L(infinity)-oriented compression, or, at most, provide a very limited number of potential L(infinity) bit-stream truncation points. We propose a new multidimensional wavelet-based L(infinity)-constrained scalable coding framework that generates a fully embedded L(infinity)-oriented bit stream and that retains the coding performance and all the scalability options of state-of-the-art L2-oriented wavelet codecs. Moreover, our codec instantiation of the proposed framework clearly outperforms JPEG2000 in L(infinity) coding sense.

Real-time delivery of video over best-effort error-prone packet networks requires scalable erasure-resilient compression systems in order to 1) meet the users' requirements in terms of quality, resolution, and frame-rate; 2) dynamically adapt the rate to the available channel capacity; and 3) provide robustness to data losses, as retransmission is often impractical. Furthermore, the employed erasure-resilience mechanisms should be scalable in order to adapt the degree of resiliency against transmission errors to the varying channel conditions. Driven by these constraints, we propose in this paper a novel design for scalable erasure-resilient video coding that couples the compression efficiency of the open-loop architecture with the robustness provided by multiple description coding. In our approach, scalability and packet-erasure resilience are jointly provided via embedded multiple description scalar quantization. Furthermore, a novel channel-aware rate-allocation technique is proposed that allows for shaping on-the-fly the output bit rate and the degree of resiliency without resorting to channel coding. As a result, robustness to data losses is traded for better visual quality when transmission occurs over reliable channels, while erasure resilience is introduced when noisy links are involved. Numerical results clearly demonstrate the advantages of the proposed approach over equivalent codec instantiations employing 1) no erasure-resilience mechanisms, 2) erasure-resilience with nonscalable redundancy, or 3) data-partitioning principles.

Due to the improvement of compression technology, limited bandwidth channels can convey more data than before. However, since compressed streams are very sensitive to transmission errors, a joint optimization of the source-channel coding is required. Unequal error protection (UEP) is the most pragmatic way to do it, and exploits the embedded nature of bitstreams produced by scalable coders. However, depending on the number of substreams and protection levels, optimization of UEP can be extremely complex. A large simplification comes from considering the contribution to the global distortion of each substream as independent from the others. This paper explores this distortion additivity assumption for JPEG2000, and shows that though the assumption does not fully hold, no visible consequence on UEP performance is introduced, while reducing significantly the complexity of the optimization process.

Error resilience is of paramount importance in video transmission over variable-bandwidth error-prone channels, such as wireless channels. In this paper, we investigate the influence of corrupted motion vectors in video coding based on motion compensated temporal filtering, and develop various error resilience and concealment mechanisms for this class of codecs. The experimental results show that our proposed motion vector coding technique significantly increases the robustness against transmission errors at the cost of less than 3% in terms of rate. It is also shown that our proposed spatial error-concealment mechanism leads to performance gains of up to 6 dB in comparison to a classical slicing-based approach employing no error concealment.

Real time delivery of video over best-effort networks requires compression systems that dynamically adapt the rate to the available channel capacity and exhibit robustness to data loss as retransmission is often impractical. Error-resilience, however, significantly lowers the coding performance when rigid design is performed based on a worst-case scenario. This paper presents an original scalable video coding scheme that couples the compression efficiency of the open-loop architecture with the robustness of multiple description source coding. The use of embedded multiple description quantization and a novel channel-aware rate-allocation allow for shaping on-the-fly the output bit-rate and the degree of resilience. As a result, robustness to data losses is traded for better visual quality when transmission occurs over reliable channels, while error-resilience is introduced when noisy links are involved. The advantage of our proposal is demonstrated in the context of packet-lossy networks.

Recently proposed scalable wavelet-based video codecs using spatial-domain motion compensated temporal filtering (SDMCTF) offer competitive compression performance when compared to H.264 and generate embedded bit-streams supporting quality, resolution and temporal scalability. To be able to support a large range of bit-rates with optimal compression efficiency, these codecs require a quality-scalable motion vector coding technique. Such an algorithm based on the integer wavelet transform followed by embedded coding of the wavelet coefficients was proposed in the recent past. In this paper, we present a quality-scalable motion vector coding algorithm using median-based motion vector prediction. The compression performance of the proposed algorithm is compared to that of the wavelet-based technique and is found to be superior. Additionally, the proposed motion vector codec is incorporated into an SDMCTF-based video codec and the benefits of using quality-scalable motion vector representations are experimentally demonstrated.

A new family of embedded multiple description scalar quantizers (EMDSQ) that support the progressive transmission of images over variable-bandwidth error-prone channels is proposed in this paper. A control mechanism that allows for tuning the redundancy between the two descriptions for each quantization level is also designed. The employed mechanism enables control of the tradeoff between the coding efficiency and error-resilience, and provides an increased robustness by improving the error resilience in the most important layers of the embedded bit-streams. Instantiations of the proposed family are incorporated in a wavelet-based embedded coding system, and the redundancy-control mechanism is practically demonstrated. Experimental results show that the proposed EMDSQ outperform the state-of-the-art multiple description uniform scalar quantizers (MDUSQ) previously proposed in the literature for error-resilient progressive image transmission.

An enhanced, off-line, segment-based rate control approach is proposed for controlling the distortion variation across successive segments of a video sequence when encoding with single-layer (MPEG-4 Baseline, MPEG-4 AVC) and wavelet video codecs. Consistent quality is achieved by a time-efficient, predictive rate-distortion modeling per segment. The individual segments are either shots or subshots, that are defined based on shot segmentation and activity analysis techniques. The rate control method solves the quality stability problem of state-of-the-art codecs, especially for less mature rate control modules (such as present in MPEG-4 AVC). The processing overhead compared to classical two-pass VBR encoding is limited while the distortion variation is significantly reduced.

Video coding techniques yielding state-of-the-art compression performance require large amount of computational resources, hence practical implementations, which target a broad market, often tend to trade-off coding efficiency and flexibility for reduced complexity. Scalable video coding instead, not only provides seamless adaptation to bit-rate variation, but also allows the end user to trim down the resources he needs to perform real-time decoding by limiting the process to a subset of the original content. Hence, by choosing the quality, frame-rate and/or resolution of the reconstructed sequence, each decoder can meet its hardware limitations without affecting the encoding process of the media provider. This paper proposes a preliminary analysis of the memory-access behavior of a fully scalable video decoder and investigates the capability of selecting the operational settings in order to adapt to the available hardware resources on the target device.

In wavelet-based L/sub /spl infin//-constrained embedded coding, the bit-stream is truncated at the bit-rate that corresponds to a guaranteed, user-defined distortion bound. The letter analyzes the optimality of embedded deadzone scalar-quantizers for high-rate L/sub /spl infin//-constrained scalable wavelet-based image coding. A rate-distortion model applicable to the family of embedded deadzone scalar-quantizers is derived and experimentally validated. Conclusions are drawn regarding the optimal subband-quantizer instantiations. The optimal quantizers are employed in a coding algorithm that retains the coding performance and the flexibility options of wavelet-based codecs while allowing for a fully embedded L/sub /spl infin//-oriented bit-stream.

MESHGRID is a novel, compact, multiscalable and animation-friendly surface representation method, which has been introduced in MPEG-4 . The MESHGRID representation attaches a description of the "global connectivity" between the vertices on the object's surface (i.e., the 3-D connectivity wireframe) to a regular 3-D grid of points (i.e., the reference grid). MESHGRID efficiently encodes the 3-D connectivity wireframe by using a new type of 3-D extension of Freeman chain-code. MESHGRID does not explicitly store the polygons of the surface, since the 3-D connectivity wireframe has particular connectivity properties allowing for the unambiguous derivation of the triangulation. The reference grid is a smooth vector field defined on a regular discrete 3-D space. This grid is efficiently compressed by using an embedded 3-D wavelet-based multiresolution intra-band coding algorithm. MESHGRID can be efficiently exploited for QoS since it allows for three types of scalability in both view-dependent and view-independent scenarios, including: 1) resolution scalability, i.e., the adaptation of the number of transmitted vertices; 2) shape precision, i.e., the adaptive reconstruction of the reference grid positions; and 3) vertex position scalability, i.e., the change of the precision of known vertex positions with respect to the reference grid. Furthermore, in addition to the classical vertex-based animation, MESHGRID also supports specific animation capabilities, such as: 1) rippling effects by changing the position of the vertices relative to corresponding reference grid points and 2) reshaping on a hierarchical basis of the regular reference grid and its attached vertices.

Summary form only given. Several methods for L/sub /spl infin//-distortion constrained compression have been proposed that target a set of fixed reconstruction-error bounds. Recently, a wavelet-based L/sub /spl infin//-constrained embedded image-coding technique was proposed that guarantees the required distortion bound while retaining the coding performance and scalability option state-of-the-art wavelet-based L/sub /spl infin//-oriented codecs. The optimality of embedded deadzone scalar-quantizers for high-rate L/sub /spl infin//-constrained scalable wavelet-based coding of images was analyzed. The optimal quantizers were employed in a coding algorithm that outperforms embedded L/sub /spl infin//-oriented wavelet-coders in terms of maximum absolute error. The algorithm was briefly described and coding results were provided.

This paper presents a novel fully-scalable wavelet video coding scheme that performs efficient open-loop motion compensated temporal filtering (MCTF) in the wavelet domain (in-band). Unlike the conventional spatial-domain MCTF (SDMCTF) schemes, which apply MCTF on the original image data and then encode the residual image using a critically-sampled wavelet transform, the framework presented here applies the in-band MCTF (IBMCTF) after the discrete wavelet transform (DWT) is performed in the spatial dimensions. To overcome the inefficiency of motion estimation (ME) in the wavelet domain, a complete-to-overcomplete DWT (CODWT) is performed. The proposed framework provides improved quality (SNR) and temporal scalability as compared with existing in-band closed-loop temporal prediction schemes with ODWT and improved spatial scalability as compared to SDMCTF. We present a thorough comparison between SDMCTF and the proposed IBMCTF in terms of coding efficiency and scalability. Furthermore, we describe several extensions that enable the filtering of the various bands to be performed independently, based on the resolution, sequence content, complexity requirements and desired scalability.

Robust progressive image transmission over unreliable channels with variable bandwidth requires multiple description coding (MDC) systems that produce highly error-resilient embedded bitstreams. The proposed embedded multiple description scalar quantizers (EMDSQ) meet the desired features consisting of a high redundancy level, fine grain rate adaptation and progressive transmission of each description. Experimental results show that EMDSQ yield better rate-distortion performance in comparison to the multiple description uniform scalar quantizers; (MDUSQ) previously proposed in the literature. Moreover, the generalized form of EMDSQ targeting an arbitrary number of channels is proposed, which offers the possibility of designing realistic coders for practical multi-channel communication systems.

It has been demonstrated recently that 3-D wavelet coding with motion-compensated temporal filtering (MCTF) provides a wide range of spatio-temporal-SNR scalability with state-of-the-art coding performance. However, the coding system is very different from the already standardized motion-compensated DCT (MC-DCT) video coders such as MPEG-2, MPEG-4, or H.26L. Nonetheless, the market acceptance of the new scalable technology will come much easier if backward compatibility to such previous standards would exist. In this paper, we present a new coder architecture where the base layer can be a standard MC-DCT coder, while the enhancement layer is an in-band MCTF codec operating in the overcomplete wavelet domain. First, we propose a simple extension of the scalable 3-D wavelet codec with a standard MC-DCT base layer. As a second step, to improve the performance of the proposed scalable coder over a wide range of bit-rates, we describe several small extensions to the standardized MC-DCT codec structure that can be applied for the base layer coding.

Recently, a new wavelet-based video codec using in-band motion compensated temporal filtering (IBMCTF) was introduced This codec is fully scalable in resolution, quality aid frame-rate. In comparison to an equivalent video coding scheme based on spatial domain motion compensated temporal filtering (SDMCTF), its compression performance when decoding to lower resolutions is very promising. However, since the IBMCTF scheme is based on in-band motion estimation, considerably more motion vector data is generated than in the SDMCTF scheme. Efficient compression of these motion vectors is therefore of utmost importance. In this paper, several solutions for the compression of motion vectors generated by a video codec based on IBMCTF are presented and compared.

The paper proposes a new framework for the control of the distortion variation in video coding schemes based on motion-compensated temporal filtering (MCTF). The distortion in an arbitrary decoded frame at any temporal level in the MCTF pyramid is expressed as a function of the distortions in the reference frames at the same temporal level. The approach is formulated for the bi-directional unconstrained MCTF (UMCTF) scheme of Turaga et al., (2002), which does not include the update-lifting step. The proposed framework can be extended to the generalized form of MCTF by utilizing additional control parameters. Experimental results demonstrate the control of the distortion variation in video coding systems based on spatial-domain and wavelet-domain MCTF. One concludes that the proposed framework provides the means of controlling the tradeoff between the average distortion and the distortion variation in each group-of-pictures (GOPs) within the decoded sequence.

Robust progressive image transmission over unreliable channels with variable bandwidth requires multiple description coding (MDC) systems that produce highly error-resilient embedded bit-streams. The proposed embedded multiple description scalar quantizers (EMDSQ) meet the desired features consisting of a high redundancy level, fine grain rate adaptation and progressive transmission of each description. Experimental results show that EMDSQ yield better rate-distortion performance in comparison to the multiple description uniform scalar quantizers (MDUSQ) previously proposed in the literature. Moreover, the generalized form of EMDSQ targeting an arbitrary number of channels is proposed, which offers the possibility of designing realistic coders for practical multi-channel communication systems.

Several techniques based on the three-dimensional (3-D) discrete cosine transform (DCT) have been proposed for volumetric data coding. These techniques fail to provide lossless coding coupled with quality and resolution scalability, which is a significant drawback for medical applications. This paper gives an overview of several state-of-the-art 3-D wavelet coders that do meet these requirements and proposes new compression methods exploiting the quadtree and block-based coding concepts, layered zero-coding principles, and context-based arithmetic coding. Additionally, a new 3-D DCT-based coding scheme is designed and used for benchmarking. The proposed wavelet-based coding algorithms produce embedded data streams that can be decoded up to the lossless level and support the desired set of functionality constraints. Moreover, objective and subjective quality evaluation on various medical volumetric datasets shows that the proposed algorithms provide competitive lossy and lossless compression results when compared with the state-of-the-art.

In this paper we elaborate on a recently proposed approach for scalable video-coding based on in-band prediction in the overcomplete wavelet domain. It is shown that, through a new calculation scheme for the level-by-level complete to overcomplete discrete wavelet transform (DWT) that exploits certain symmetries, important reductions in the multiplication budget are obtained in comparison to the fastest-known algorithm of the literature. Based on the derived overcomplete transform-domain coefficients, a pixel-accurate motion estimation and compensation (ME/MC) algorithm is proposed, which provides a hybrid-coding framework that supports full scalability in resolution, quality and frame rate. To give an indication of the coding performance of such a system, some preliminary results are reported.

A new in-band motion compensation algorithm for wavelet-based video coding is proposed: the bottom-up prediction algorithm (BUP). The BUP algorithm overcomes the periodic shift-invariance of the discrete wavelet transform (DWT) and is formalized into new prediction rules using filtering operations. BUP is based on the relationships between the subbands of the shifted input signal and the subbands of the non-shifted reference signal, whereby the number of shifts is limited by the periodic shift-invariance of the DWT. We derive the algorithm for the 1-D DWT with three decomposition levels. The combination of all prediction rules of the BUP algorithm defines a new transform: the bottom-up overcomplete DWT or BUP ODWT, which is shift-invariant. The BUP ODWT calculates the overcomplete subbands by applying the prediction rules to the critically sampled subbands of a wavelet-transformed image. The envisaged application for the BUP algorithm is spatially scalable video coding.

This paper assesses the coding performance of several state-of-the-art 3D wavelet coders and compares them with newly proposed compression methods exploiting quadtree and block-based coding concepts, layered zero-coding principles and context-based arithmetic coding. The discussed wavelet-based coding algorithms produce losslessly compressed embedded data streams, supporting the desired progressive data transmission functionality. Moreover, objective and subjective quality evaluations on various medical volumetric datasets show that the newly proposed algorithms provide competitive lossy and lossless compression results compared to the state-of-the-art.

MESHGRID is a novel, compact, multi-scalable and animation-friendly surface representation method, which has been introduced in MPEG-4. The MESHGRID representation attaches a description of the "global connectivity" between the vertices on the object's surface (i.e. the 3D connectivity wireframe) to a regular 3D grid of points (i.e. the reference-grid). The 3D connectivity wireframe is efficiently encoded by using a new type of 3D extension of Freeman chain-code. MESHGRID does not explicitly store the polygons of the surface, since the 3D connectivity wireframe has particular connectivity properties allowing for the unambiguous derivation of the triangulation. The reference-grid is a smooth vector field defined on a regular discrete 3D space. This grid is efficiently compressed by using an embedded 3D wavelet-based multi-resolution intra-band coding algorithm. MESHGRID allows for three types of scalability in both view-dependent and view-independent scenarios: resolution scalability, shape precision, and vertex position scalability. Furthermore, in addition to the classical vertex-based animation, MESHGRID supports specific animation capabilities, such as rippling effects and reshaping on a hierarchical basis of the regular reference-grid and its attached vertices.

This paper describes an efficient implementation of an image coding system based on the independent wavelet-tree coding concept. The system consists of a transform and a (de)coding engine that operate in a pipelined fashion. The main focus of this paper is on the encoding part since, due to the system architecture, the decoder has identical memory utilization. Experimental results prove that the proposed system achieves comparable coding performance to the state-of-the-art, while it localizes the memory accesses to small memory modules and uses minimal computational resources.

A new method for the implementation of the binary-tree decomposition of the convolution-based wavelet transform, called the local wavelet transform (LWT) has been recently proposed in the literature. While it produces exactly the same results as the classical row-column implementation of the transform, it has many implementation benefits. This fact is shown experimentally for the first time for a general-purpose processor-based architecture, by comparing our C implementation of the LWT with an optimal C implementation of the lifting-scheme row-column algorithm. The comparisons are made for the forward multilevel binary-tree decomposition using the 9/7 filter pair, in the typical Intel Pentium processor family.

A new region-based compression scheme for color images is proposed. The segmentation method belongs to the split and merge category. Splitting is carried out using the watershed transform. In the merging stage, a fuzzy color preserving rule-based system is introduced to express the dissimilarity between two regions. The compression part is based on the shape adaptive DCT with /spl Delta/DC correction method. The quantization matrices have been designed according to the properties of the used transform. This method yields promising results, mostly attributed to the efficiency of the segmentation and particularly the fuzzy dissimilarity estimation.

A major problem in video surveillance is the robust detection of intrusive motion. A widespread technique is frame differencing, initiating an alarm procedure when the difference exceeds a predefined threshold. However, background motion and conflict motion are hard to distinguish. Full-search (FS) block-based motion estimation can be made more robust with respect to the rejection of very small motions, but it is still too sensitive and it has also the disadvantage of the high computational complexity. In our proposal we suggest to use a detection technique that is based on a new center-biased search technique and an original adaptive motion classification system. The proposed approach was validated via an implementation on a very long instruction word (VLIW) processor.