Methods for Optical Flow (OF) estimation based on Deep Learning have considerably improved traditional approaches in challenging and realistic conditions. However, data-driven approaches can inherently be biased, leading to unexpected under-performance in real application scenarios. In this paper, we first observe that the OF estimation accuracy varies with motion direction, and name this phenomenon ‘OF sign imbalance’. The sign imbalance cannot be assessed by means of the endpoint-error (EPE), the typical training and evaluation metric for Deep Optical Flow estimators. This paper tackles this issue by proposing a new metric to assess the sign imbalance, which is compared to the endpoint-error. We provide an extensive evaluation of the sign imbalance for the state-of-the-art optical flow estimators. Based on the evaluation, we propose two strategies to mitigate the phenomenon, i) by constraining the model estimations during inference, and, ii) by constraining the loss function during training. Testing and training code is available at: www.github.com/stsavian/equivariant_of_estimation.

The emerging of depth-camera technology is paving the way for a variety of new applications and it is believed that plane detection is one of them. In fact, planes are common in man-made living structures, thus their accurate detection can benefit many visual-based applications. The use of depth information allows detecting planes characterized by complex pattern and texture, where the texture-based plane detection algorithms usually fail. In this paper, we propose a robust depth-driven plane detection (DPD) algorithm which consists of two parts: the growing-based plane detection and a two-stage refinement. The proposed approach starts from the seed patch with the highest planarity and uses the estimated equation of the growing plane and a dynamic threshold function to steer the growing process. Aided with this mechanism, each seed patch can grow to its maximum extent, and then the next seed patch starts to grow. This process is iteratively repeated so as to detect all the planes. Moreover, the refinement is proposed to tackle two common problems suffered by growing-based approaches, the over-growing problem, and the under-growing problem. Validated by extensive experiments, the proposed DPD algorithm is able to accurately detect planes and robust to various testing conditions. In terms of applications, it can be used as the pre-processing step for a variety of applications, such as, planar object recognition, super-resolution of the time-of-flight depth images with intrinsically low resolution.

Interframe and interview predictions are widely employed in multiview video coding. This technique improves the coding efficiency, but it also increases the vulnerability of the coded bitstream. Thus, one packet loss will affect many subsequent frames in the same view and probably in other referenced views. To address this problem, a region-based multiple description coding scheme is proposed for robust 3-D video communication in this paper, in which two descriptions are formed by setting the left and right view as dominant in the first and second description, respectively. This approach exploits the fact that most regions in the reference view could be synthesized from the base view. Hence, these regions could be skipped or only coarsely encoded. In our work, the disoccluded regions, illumination-affected regions, and remaining regions are first determined and extracted. By assigning different quantization parameters for these three different regions according to the network status, an efficient multiple description scheme is formed. Experimental results demonstrate that the proposed scheme achieves considerably better performance compared with the traditional approach.

Multiview video streaming continues to gain popularity due to the great viewing experience it offers, as well as its availability that has been enabled by increased network throughput and other recent technical developments. User demand for interactive multiview video streaming that provides seamless view switching upon request is also increasing. However, it is a highly challenging task to stream stable and high quality videos that allow real-time scene navigation within the bandwidth constraint. In this paper, a convolutional neural network (ConvNet)-assisted seamless multiview video streaming system is proposed to tackle the challenge. The proposed method solves the problem from two perspectives. First, a ConvNet-assisted multiview representation method is proposed, which provides flexible interactivity without compromising on multiview video compression efficiency. Second, a bit allocation mechanism guided by a navigation model is developed to provide seamless navigation and adapt to network bandwidth fluctuations at the same time. These two blocks work closely to provide an optimized viewing experience to users. They can be integrated into any existing multiview video streaming framework to enhance overall performance. Experimental results demonstrate the effectiveness of the proposed method for seamless multiview streaming.

In video coding, traditional motion estimation methods work well for videos with camera translational motion, but their efficiency drops for other motions, such as rotational and dolly motions. In this paper, a motion-information-based three-dimensional (3D) video coding method is proposed for texture plus depth 3D video. The synchronized global motion information of the camera is obtained to assist the encoder improve its rate-distortion performance by projecting the temporal neighboring texture and depth frames into the position of the current frame, using the depth and camera motion information. Then, the projected frames are added into the reference buffer list as virtual reference frames. As these virtual reference frames could be more similar to the current to-be-encoded frame than the conventional reference frames, the required bits to represent the residual will be reduced. The experimental results demonstrate that the proposed scheme enhances the coding performance for all camera motion types and for various scene settings and resolutions using H.264 and HEVC standards, respectively. With the computer graphic sequences, for H.264, the average gain of texture and depth coding are up to 2 dB and 1 dB, respectively. For HEVC and HD resolution sequences, the gain of texture coding reaches 0.4 dB. For realistic sequences, up to 0.5 dB gain (H.264) is achieved for the texture video, while up to 0.7 dB gain is achieved for the depth sequences.

Large amount of redundant information and huge data size have been a serious problem for multiview video systems. To address this problem, one popular solution is mixed-resolution, where only few viewpoints are kept with full resolution and other views are kept with lower resolution. In this paper, we propose a super-resolution (SR) method, where the low-resolution viewpoints in the 3D video are up-sampled using a fully convolutional neural network. By simply projecting the neighboring high resolution image to the position of the low resolution image, we learn the relationship of high and low resolution patches, and reconstruct the low resolution images into high resolution ones using the projected image information. We propose to use a fully convolutional neural network to establish a mapping between those images. The network is barely trained on 17 pairs of multiview images, and tested on other multiview images and video sequences. It is observed that our proposed method outperforms existing methods objectively and subjectively, with more than 1 dB average gain achieved. Meanwhile, our network training procedure is efficient, with less than 3 hours using one Titan X GPU.

In multi-view video plus depth (MVD) format, virtual views are synthesized by the compressed texture videos and their associated depth through depth-image-based rendering. In this paper, we consider the setup where both the encoded texture and depth bitstreams experience packet losses during transmission. Different packetization strategies are investigated and a novel strategy is developed to improve error resilience of MVD-based video transmission, where texture data and its corresponding depth are put into the same packet. The size of texture plus associated depth data included in each packet needs to be less than the Maximum Transfer Unit (MTU). Experimental results demonstrate that our proposed packetization scheme yields a significant improvement in terms of both texture views and synthesized virtual views quality when fit in H.264/AVC.

A 3-D multiview video gives users an experience that is different from that provided by a traditional video; however, it puts a huge burden on limited bandwidth resources. Mixed-resolution video in a multiview system can alleviate this problem by using different video resolutions for different views. However, to reduce visual uncomfortableness and to make this video format more suitable for free-viewpoint television, the low-resolution (LR) views need to be super-resolved to the target full resolution. In this paper, we propose a virtual-view-assisted super-resolution algorithm, where the inter-view similarity is used to determine whether the missing pixels in the super-resolved frame need to be filled by virtual-view pixels or by spatial interpolated pixels. The decision mechanism is steered by the texture characteristics of the neighbors of each missing pixel. Furthermore, the inter-view similarity is used, on the one hand, to enhance the quality of the virtual-view-copied pixels by compensating the luminance difference between different views and, on the other hand, to enhance the original LR pixels in the super-resolved frame by reducing their compression distortion. Thus, the proposed method can recover the details in regions with edges while maintaining good quality at smooth areas by properly exploiting the high-quality virtual-view pixels and the directional correlation of pixels. The experimental results demonstrate the effectiveness of the proposed approach with a peak signal-to-noise ratio gain of up to 3.85 dB.

In this paper, we propose a depth map down-sampling and coding scheme that minimizes the view synthesis distortion. Moreover, a solution for the optimal depth map down-sampling problem that minimizes the depth-caused distortion in the virtual view by exploiting the depth map and the associated texture information along with the up-sampling method to be used in the decoder side is derived. Furthermore, to enhance compression performance, the synthesized view distortion, which is evaluated by emulating the interpolation and the virtual view synthesis process, is used in the optimization objective function for coding mode selection in the video encoder. Experimental results show that both the proposed depth map down-sampling and encoding methods lead to good performance, and the average bit rate reduction is 2.62% compared with 3D-AVC.

High Efficient Video Coding (HEVC) standard has achieved significant rate-distortion improvement over the previous standard H.264/AVC. However, the complexity that comes from its flexible data structure representation is an obstacle for its wide application. To reduce the overall complexity and encoding time, this paper proposes a statistical approach for motion estimation skipping (SAMEK). The SAMEK method avoids some unnecessary motion estimations in units with less probability of being referenced. These units can be recognized by the two rules of SAMEK method, namely ZeroCase and DecreaseCase. These two rules are summarized by statistically analyzing the relationships between each PU and its references. The experimental results demonstrate that our method can save up to 9.5% encoding time (averagely 6.87%) with negligible rate-distortion losses (averagely 0.006 dB) when compared with HEVC encoder with Test Zone search (TZ search) enabled.

In this paper, a motion-information-based 3D video coding method is proposed for the texture plus depth 3D video format. The synchronized global motion information of camcorder is sampled to assist the encoder to improve its rate-distortion performance. This approach works by projecting temporal previous frames into the position of the current frame using the depth and motion information. These projected frames are added in the reference buffer as virtual reference frames. As these virtual reference frames are more similar to the current frame than the conventional reference frames, the required residual information is reduced. The experimental results demonstrate that the proposed scheme enhances the coding performance in various motion conditions including rotational and translational motions.

Due to the prediction structures employed in video coding, the loss of one packet will affect many following frames. In this paper, a multiple description coding scheme with stagger frame order is proposed for stereoscopic 3-D videos. First, the reference and auxiliary views in stereoscopic sequences will be asymmetrically encoded into one description, whereas the other description will be formed in the same way with one dumb frame delay. Because of the stagger frame order, the coarsely encoded B frames will be inserted into different positions of the two descriptions. If a certain frame encoded with I/P mode is lost, then its corresponding B-frame version will be employed to compensate for the loss. In each description, the quantization steps of B frames are tuned based on a closed-form solution that considers the video contents, network status, frame positions in the group of picture, and the layer of the views. For further improvement, a fusing scheme is provided. The experimental results demonstrate that the proposed scheme outperforms state-of-the-art schemes. Specifically, up to 1.3-dB gain is achieved in the case of packet loss, and 2-dB gain is obtained for the side/central performance.

In the multiview video plus depth (MVD) coding format, both texture and depth views are jointly compressed to represent the 3-D video content. The MVD format enables synthesis of virtual views through depth-image-based rendering; hence, distortion in the texture and depth views affects the quality of the synthesized virtual views. Bit allocation between texture and depth views has been studied with some promising results. However, to the best of our knowledge, most of the existing bit-allocation methods attempt to allocate a fixed amount of total bit rate between texture and depth views; that is, to select appropriate pair of quantization parameters for texture and depth views to maximize the synthesized view quality subject to a fixed total bit rate. In this paper we propose a scalable bit-allocation scheme, where a single ordering of texture and depth packets is derived and used to obtain optimal bit allocation between texture and depth views for any total target rates. In the proposed scheme, both texture and depth views are encoded using the quality scalable coding method; that is, medium grain scalable (MGS) coding of the Scalable Video Coding (SVC) extension of the Advanced Video Coding (H.264/AVC) standard. For varying target total bit rates, optimal bit truncation points for both texture and depth views can be obtained using the proposed scheme. Moreover, we propose to order the enhancement layer packets of the H.264/SVC MGS encoded depth view according to their contribution to the reduction of the synthesized view distortion. On one hand, this improves the depth view packet ordering when considered the rate-distortion performance of synthesized views, which is demonstrated by the experimental results. On the other hand, the information obtained in this step is used to facilitate optimal bit allocation between texture and depth views. Experimental results demonstrate the effectiveness of the proposed scalable bit-allocation scheme for texture and depth views.

Reed-Solomon erasure code is one of the most studied protection methods for video streaming over unreliable networks. As a block-based error correcting code, large block size and increased number of parity packets will enhance its protection performance. However, for video applications this enhancement is sacrificed by the error propagation and the increased bitrate. So, to tackle this paradox, we propose a rate-distortion optimized redundancy allocation scheme, which takes into consideration the distortion caused by losing each slice and the propagated error. Different from other approaches, the amount of introduced redundancy and the way it is introduced are automatically selected without human interventions based on the network condition and video characteristics. The redundancy allocation problem is formulated as a constraint optimization problem, which allows to have more flexibility in setting the block-wise redundancy. The proposed scheme is implemented in JM14.0 for H.264, and it achieves an average gain of 1dB over the state-of-the-art approach.

Planar surface is a common feature in man-made structure, thus accurate detection of planar surface can benefit the image/video segmentation and reconstruction and also the navigation system of robots. Since depth map represents the distance from one object to the capturing camera in a grey image, it also can represent the surface characteristics of the objects. So in this paper, we propose a novel Depth-map Driven Planar Surface Detection (DDPSD) method, where detection starts from "the most flat" seed patch on the depth map and uses dynamic threshold value and surface function to control the growing process. Compared with one of the popular planar surface detection algorithms, RANdom SAmples Consensus (RANSAC), the accuracy of the proposed method is obviously superior on typical indoor scenes. Moreover, semi-planar surfaces can be also successfully detected by the proposed method.

In this paper, an MDC scheme based on forward error correction(FEC) within expanding windows is proposed. Firstly, the video sequence will be coded into source packets with/without slice group enabled. Secondly, the appropriate FEC packets are inserted according to the packet loss rate. Since the previous frames in a GOP is generally more important than the following frames in the GOP, an expanding window is exploited so that the FEC packets for the current frame will also protect the previous frames in the window. After this, the source packets with the inserted FEC packets will be divided into two descriptions and transmitted into two independent channels. When some packets in one description are lost, FEC decoding will try to recover the lost packets. Through this scheme, the source packets can get appropriate protection while the compression efficiency will not be degraded too much. The experimental results show that the proposed scheme outperforms the compared schemes up to 3 dB.

Forward error correction (FEC) codes are widely studied to protect streamed video over unreliable networks. Typically, enlarging the FEC coding block size can improve the error correction performance. For video streaming applications, this could be implemented by grouping more than one video frame into one FEC coding block. However, in this case, it leads to decoding delay, which is not tolerable for real-time video streaming applications. In this paper, to solve this dilemma, a real-time video streaming scheme using randomized expanding Reed-Solomon (RS) code is proposed. In this scheme, the RS coding block includes not only the video packets of the current frame, but could also include all the video packets of previous frames in the current group of pictures. At the decoding side, the parity-check equations of the current frame are jointly solved with all the parity-check equations of the previous frames. Since video packets of the following frames are not encompassed in the RS coding block, no delay will be caused for waiting for the video or parity packets of the following frames both at encoding and decoding sides. Experimental results show that the proposed scheme outperforms other real-time error resilient video streaming approaches significantly, specifically, for the Foreman sequence, the proposed scheme could provide 1.5 dB average gain over the state-of-the-art approach for 10% i.i.d. packet loss rate, whereas for the burst loss case, the average gain is more than 3 dB.MATLAB code of this paper is available for download at http://www.mmtlab.com.

For real-time video applications, such as video telephony service, the allowed maximum end-to-end transmission delay is usually fixed. The packets arriving at the destination out of the maximum end-to-end delay are treated as late-arrival packets, and these packets are discarded in traditional video transmission systems. In this paper, in order to improve the system performance, we propose to exploit these packets to update the decoder reference buffer. Two schemes are proposed to exploit the late-arrival packets, one scheme is to use sliding-window updating, where the updating window is moving; another scheme is to use fixed-window update together with systematic Reed-Solomon code. The effectiveness of the two schemes are validated by simulation results without adding extra delay. It is found that in both schemes, the updating window size plays an important role on the system performance.

Reed-Solomon erasure codes are commonly studied as a method to protect the video streams when transmitted over unreliable networks. As a block-based error correcting code, on one hand, enlarging the block size can enhance the performance of the Reed-Solomon codes; on the other hand, large block size leads to long delay which is not tolerable for real-time video applications. In this paper a novel Dynamic Sub-GOP FEC (DSGF) approach is proposed to improve the performance of Reed-Solomon codes for video applications. With the proposed approach, the Sub-GOP, which contains more than one video frame, is dynamically tuned and used as the RS coding block, yet no delay is introduced. For a fixed number of extra introduced packets, for protection, the length of the Sub-GOP and the redundancy devoted to each Sub-GOP becomes a constrained optimization problem. To solve this problem, a fast greedy algorithm is proposed. Experimental results show that the proposed ap proach outperforms other real-time error resilient video coding technologies.

When the video streams are transmitted over the unreliable networks, forward error correction (FEC) codes are usually used to protect them. Reed-Solomon codes are block-based FEC codes. On one hand, enlarging the block size can enhance the performance of the Reed- Solomon codes. On the other hand, large Reed-Solomon block size leads to long delay which is not tolerable for real-time video applications. In this paper a novel approach is proposed to improve the performance of Reed-Solomon codes. With the proposed approach, more than one video frame are encompassed in the Reed-Solomon coding block yet no delay is introduced. Experimental results show that the proposed approach outperforms other real-time error resilient video coding technologies.

In this paper, a novel multiple description video coding scheme is proposed to insert and control the redundancy at macro block (MB) level. By analyzing the error propagation paths, the relative importance of each MB is determined. The paths, in practice, depend on both the video content and the adopted video coder. Considering the relative importance of the MB and the network status, an unequal protection for the video data can be realized to exploit the redundancy effectively. In addition, a simple and effective approach is introduced to tune the quantization parameter for the variable rate coding case. The whole scheme is implemented in H.264/AVC by employing its coding options, thus generating descriptions that are compatible with the baseline profile and extended profile of H.264/AVC. Due to its general property, the proposed approach can be employed for other hybrid video codecs. The results demonstrate the advantage of the proposed approach over other H.264/AVC multiple description schemes.

In this paper, we devise a procedure that mimics the behavior of a progressive video stream starting from a non progressive one such as H.264/AVC encoded video. This allows one to unequally protect the video data in an efficient way, according to their importance and the network state. The reported results demonstrate the superior performance of the proposed approach in comparison to state-of-the-art methods for resilient transmission of H.264/AVC data. Moreover, the flexibility in terms of redundancy insertion and achieved quality levels, allows one to span different applications, possibly including P2P video streaming.

In this letter, we present a performance comparison between multiple description coding (MDC) and unequal loss protection (ULP) for progressive image transmission over lossy packet networks. Two optimization criteria are considered, i.e., a multi-quality criterion, whenNdistinct quality levels are guaranteed at the decoder side, and the optimization of the expected quality at the receiver. We resort to both a semi-analytical approach and simulation results. To enable numerical comparisons, we address a specific MDC algorithm suitable for progressive imaging, and a state-of-the-art ULP algorithm based on Reed Solomon codes. The results, although cannot be generalized to any MDC and ULP methods, are useful to put into evidence some general features that can drive the selection of the most proper technique for the application at hand. In fact, they allow to put into evidence the main advantages and drawbacks of either technique.

Digital fountain codes have emerged as a low-complexity alternative to Reed-Solomon codes for erasure correction. The applications of these codes are relevant especially in the field of wireless video, where low encoding and decoding complexity is crucial. In this paper, we introduce a new class of digital fountain codes based on a sliding-window approach applied to Raptor codes. These codes have several properties useful for video applications, and provide better performance than classical digital fountains. Then, we propose an application of sliding-window Raptor codes to wireless video broadcasting using scalable video coding. The rates of the base and enhancement layers, as well as the number of coded packets generated for each layer, are optimized so as to yield the best possible expected quality at the receiver side, and providing unequal loss protection to the different layers according to their importance. The proposed system has been validated in a UMTS broadcast scenario, showing that it improves the end-to-end quality, and is robust towards fluctuations in the packet loss rate.

Multiple description coding (MDC) makes use of redundant representations of multimedia data to achieve resiliency. Descriptions should be generated so that the quality obtained when decoding a subset of them only depends on their number and not on the particular received subset. In this paper, we propose a method based on the principle of encoding the source at several rates, and properly blending the data encoded at different rates to generate the descriptions. The aim is to achieve efficient redundancy exploitation, and easy adaptation to different network scenarios by means of fine tuning of the encoder parameters. We apply this principle to both JPEG 2000 images and H.264/AVC video data. We consider as the reference scenario the distribution of contents on application-layer overlays with multiple-tree topology. The experimental results reveal that our method favorably compares with state-of-art MDC techniques.

In this paper we address a novel scheme to protect video sequences according to slice importance based on slice reordering, ULP and error-concealment techniques. The approach does not require the modification to the video decoder although an application-layer channel coding is required. Simulation results show that the proposed algorithm outperforms state-of-the-art approaches, reducing the gap with the upper-bound error-free performance curve. Moreover, the complexity of the additional stage required to pilot the protection allocation stage is negligible with respect to traditional ULP schemes.

In this letter, we propose a novel unequal loss protection scheme, which allocates FEC codes to video slices according to their impact on the GOP distortion. This is evaluated taking the concealment procedure and the drift effect into account. Simulation results show that the proposed algorithm outperforms state-of-the-art approaches, reducing the gap with the error-free performance curve. Moreover, the complexity of the additional stage required to pilot the protection allocation stage is negligible with respect to traditional ULP schemes.

We propose an algorithm that improves the performance of rate-distortion-based multiple description coding (RD-MDC). The gain is particularly significant in the high redundancy region, where RD-MDC suffers a major performance penalty with respect to MDC bounds. The improvement is obtained with negligible additional computational cost, by exploiting the coarse information also at the central decoder. The proposed method can be generalized to all MDC schemes that use scalar quantization, without modifying the quantizer structure. This feature guarantees the generation of descriptions that can be decoded without any modification of the decoder.

In this paper, a novel H.264 multiple description technique is proposed. The coding approach is based on the redundant slice representation option, defined in the H.264 standard. In presence of losses, the redundant representation can be used to replace missing portions of the compressed bit stream, thus yielding a certain degree of error resilience. This paper addresses the creation of two balanced descriptions based on the concept of redundant slices, while keeping full compatibility with the H.264 standard syntax and decoding behavior in case of single description reception. When two descriptions are available still a standard H.264 decoder can be used, given a simple preprocessing of the received compressed bit streams. An analytical setup is employed in order to optimally select the amount of redundancy to be inserted in each frame, taking into account both the transmission condition and the video decoder error propagation. Experimental results demonstrate that the proposed technique favorably compares with other H.264 multiple description approaches.

In this paper a novel H.264 multiple description technique is proposed. The coding approach is based on the redundant slice representation option, defined in the H.264 standard. In presence of losses, the redundant representation can be used to replace missing portions of the compressed bitstream, thus yielding a certain degree of error resilience. This paper addresses the creation of two balanced descriptions based on the concept of redundant slices, while keeping full compatibility with the H.264 standard syntax and decoding behavior. Moreover, a practical algorithm for redundancy tuning as a function of the packet loss rate is provided. Experimental results demonstrate that the proposed technique favorably compares with other H.264 multiple description approaches.

We propose a new lossy compression algorithm for hyperspectral images, which is based on the spectral Karhunen-Loeve transform, followed by spatial JPEG 2000, which employs a model of anomalous pixels during the compression process. Results on Airborne Visible/Infrared Imaging Spectrometer scenes show that the new algorithm provides better rate-distortion performance, as well as improved anomaly detection performance, with respect to the state of the art.

Multiple description coding (MDC) is a good way to combat packet losses in error-prone networks subject to packet erasures. However, redundancy tuning is often difficult, and this makes the generation of descriptions with good redundancy-rate-distortion performance a hard job. Moreover, the complexity of generating more than two descriptions represents a strong limitation to MDC. In this letter, we propose a method that exploits the data rate-distortion characteristics to generate multiple descriptions for JPEG 2000 with tunable redundancy levels. The identification of the best number of descriptions as a function of the network conditions is also addressed

In this letter, the mismatch error due to the replacement of a fine with a coarse quantizer is considered, and an analytical model is proposed to describe the related distortion. Simulations show that this model is highly accurate and can be used to estimate the expected distortion of DPCM-based codecs in order to better allocating the rate. For highly correlated sources, this leads to a gain of 1 to 1.5 dB over an exhaustive search method that adopts a uniform redundancy allocation. Moreover, it permits to allocate the redundancy by an easy-to-solve analytical model of the system.

Transform-based lossy compression has a huge potential for hyperspectral data reduction. Hyperspectral data are 3-D, and the nature of their correlation is different in each dimension. This calls for a careful design of the 3-D transform to be used for compression. In this paper, we investigate the transform design and rate allocation stage for lossy compression of hyperspectral data. First, we select a set of 3-D transforms, obtained by combining in various ways wavelets, wavelet packets, the discrete cosine transform, and the Karhunen–Loève transform (KLT), and evaluate the coding efficiency of these combinations. Second, we propose a low-complexity version of the KLT, in which complexity and performance can be balanced in a scalable way, allowing one to design the transform that better matches a specific application. Third, we integrate this, as well as other existing transforms, in the framework of Part 2 of the Joint Photographic Experts Group (JPEG) 2000 standard, taking advantage of the high coding efficiency of JPEG 2000, and exploiting the interoperability of an international standard. We introduce an evaluation framework based on both reconstruction fidelity and impact on image exploitation, and evaluate the proposed algorithm by applying this framework to AVIRIS scenes. It is shown that the scheme based on the proposed low-complexity KLT significantly outperforms previous schemes as to rate-distortion performance. As for impact on exploitation, we consider multiclass hard classification, spectral unmixing, binary classification, and anomaly detection as benchmark applications.

In this paper, a novel multiple description coding technique is proposed, based on optimal Lagrangian rate allocation. The method assumes the coded data consists of independently coded blocks. Initially, all the blocks are coded at two different rates. Then blocks are split into two subsets with similar rate distortion characteristics; two balanced descriptions are generated by combining code blocks belonging to the two subsets encoded at opposite rates. A theoretical analysis of the approach is carried out, and the optimal rate distortion conditions are worked out. The method is successfully applied to the JPEG 2000 standard and simulation results show a noticeable performance improvement with respect to state-of-the art algorithms. The proposed technique enables easy tuning of the required coding redundancy. Moreover, the generated streams are fully compatible with Part 1 of the standard.

Multiple description coding can be implemented as pre- and postprocessing to all standards for image and video communications, with obvious advantages. This can be achieved by generating two subsets from the original data; a controllable amount of extra redundancy between the descriptions has to be inserted to help the estimation of the possibly lost description from the received one. This redundancy can be in the form of spatial oversampling. In this paper, we propose and develop a mathematical framework for two descriptions pre- and postprocessing methods, which exploit the correlation characteristics of the visual data in order to better implement the multiple description coding paradigm. Simulation results show a noticeable performance improvement of both the proposed methods with respect to state-of-the art algorithms, in terms of both rate/redundancy/distortion tradeoff and computational complexity.

In order to store, and retrieve images from large databases, we propose a framework, based on multiple description coding paradigms, that disseminates images over distributed servers. Consequently, decentralized download can be performed, thus reducing links overload and hotspot areas without penalizing downloads speed. Moreover, the tradeoff between system reliability and storage requirement can be achieved by tuning descriptions redundancy, thus providing high flexibility in terms of storage resources, reliability of access, and performance. The scalability of the proposed framework is achieved by the intrinsic progressivity of the multiple description schemes. Moreover, we demonstrate that system can work properly regardless of server crashes

In this letter we propose a new technique for progressive coding of hyperspectral data. Specifically, we employ a hybrid three-dimensional wavelet transform for spectral and spatial decorrelation in the framework of Part 2 of the JPEG 2000 standard. Both onboard and on-the-ground compression are addressed. The resulting technique is compliant with the JPEG 2000 family of standards and provides competitive performance with respect to state-of-the-art techniques.

Despite multiple description coding (MDC) paradigm is effective to combat packet losses on networks, it is not included in any multimedia co-decoding standard. This can be justified by the complexity of the redundancy tuning task of many of the proposed methods. Moreover, the complexity of generating more than two descriptions represents a strong limitation to MDC. In this paper, we generalize the MDC scheme proposed for JPEG 2000 in T. Tillo and G. Olmo (2004) to more than two descriptions. Using this flexible MDC scheme, we show the importance of properly tuning both the redundancy and the number of descriptions to match the network condition. Finally, we derive the conditions in order to determine when it is better to use single description coding or MDC

Hyperspectral image compression has recently attracted a remarkable interest for remote sensing applications. In this paper we propose a unified embedded lossy-to-lossless compression framework based on the JPEG 2000 standard. In particular, we exploit the multicomponent transformation feature of Part 2 of JPEG 2000 to devise a compression framework based on a spectral decorrelating transform followed by JPEG 2000 compression of the transformed coefficients. We evaluate several possible choices for the spectral transform, including a floating-point DCT, an integer DCT, and a wavelet transform. The final version of the proposed algorithm has been compared to 3D-SPIHT in the lossy case, and to several state-of-the-art compression algorithms including JPEG-LS and 3D-CALIC in the lossless case. Experimental results on AVIRIS data show that the proposed technique exhibits very competitive performance for both reversible and irreversible compression, with significantly lower complexity than DPCM-based methods, and memory requirements compatible with typical onboard processing subsystems of remote sensing platforms.

When transmitting multimedia information over non-prioritized networks subject to packet losses, multiple description coding with an arbitrary number of descriptions is an effective choice in order to minimize the end-to-end distortion. Such descriptions should be generated so that the quality obtained decoding a subset of them depends only on their number and not on the particular received subset. In this paper, we propose an encoding procedure to generate an arbitrary number of balanced descriptions using a multi-rate allocation scheme which exploits the R-D characteristic of the data

This work addresses the coding of 3-dimensional scenes, as captured by distributed vision sensors with catadioptric cameras. Spherical images allow for avoiding distortion due to the common euclidian assumption in the representation of the plenoptic function. We consider here low complexity encoding of the sensor outputs, in a framework where the cameras could be placed anywhere in the scene, and where the sensors do not communicate to each other. Since multiple spherical images of the same scene most probably provide a redundant representation, we propose to have different compression ratios for different cameras, in order to reduce the overhead of information. The decoder performs a joint decoding of the multiples images, by motion estimation, and joint refinement by consistent inverse quantization. It is finally shown that, even in the absence of any information about the scene or the position of the cameras, the proposed scheme offers improved performance with respect to an independent encoding of the spherical images, especially at low coding rate

In the multiple description paradigm, a controllable amount of redundancy is inserted among descriptions, in order to help estimating those ones which are possibly lost due to network congestion. This redundancy can also be exploited in order to correct errors at bit level. In this paper, we propose a novel technique to generate multiple descriptions of video encoded with motion-JPEG 2000, which exploits the inserted extra redundancy also to guarantee error protection in case all descriptions are received, but are possibly affected by bit errors. This method yields excellent performance, since it guarantees not only protection of video information transmitted over non prioritized networks subject to independent packet erasure processes, but also resilience towards the corruption at bit level. Moreover, the generated streams are fully compatible with the part 3 of the JPEG 2000 standard.

Nowadays, wireless multimedia applications are experiencing a rapid growth; in this scenario, challenging obstacles, such as packet losses due to congestion and band limitation along with bit-level error corruption, require the design of novel solutions for robust multimedia delivery. New standards for multimedia applications are incorporating many tools for error resilience; as an example, JPEG 2000 part 11 is explicitly devoted to the wireless applications of the image coding standard. In this paper we address a novel multiple description coding technique, based on post-processing rate allocation of embedded bitstreams. The proposed approach is compliant with the JPEG 2000 standard and has the ability to jointly cope with both packet losses and bit errors. Experimental results show that the designed algorithm significantly outperforms other techniques based on unequal error protection by means of RS codes.

We propose a novel technique to generate rate-distortion optimized multiple descriptions of images, exploiting the rate-allocation strategy embedded in the JPEG2000 encoder. The proposed scheme can be applied to any encoding algorithm, given that the rate allocation is based on code-block truncation. The method yields excellent performance in terms of both central and side distortion, outperforming state-of-the art techniques. Moreover, the single description decoding is fully compatible with the JPEG2000 Part 1 decoder.