The Consultative Committee for Space Data Systems (CCSDS) first standardized near-lossless coding capabilities in the CCSDS 123.0-B-2 algorithm. However, this standard does not describe strategies to produce high-throughput hardware implementations, which are not trivial to derive from its definition. At the same time, throughput optimizations without significant compression performance penalty are paramount to enable real-time compression on-board next-generation satellites. This work demonstrates that the weight update stage of the CCSDS 123.0-B-2 predictor can be selectively bypassed to enhance throughput for both lossless and near-lossless modes with minimal impact on compression performance and still produce fully compliant bitstreams. Skipping the weight update implies that those weights must be carefully chosen outside the original CCSDS 123.0-B-2 pipeline. Two strategies are proposed to select effective weight values based on whether a priori information about the current image is exploited or not. Comprehensive experimental results are presented for both proposed strategies and for lossless and near-lossless regimes, using a representative set of hyperspectral images. The coding penalty is, on average, 1% for lossless and 8% for near-lossless, depending on the strategy used to set the initial weights. The proposed method obtains a maximum throughput of 1 processed sample per clock cycle when it is evaluated using High-Level Synthesis (HLS), consuming 4.6% of the LUTs and 31.1% of the internal memory on a Xilinx Kintex UltraScale space-grade FPGA.

This letter proposes methods to improve data size and access time for$k^{2}$-raster, a losslessly compressed data structure that provides efficient storage and real-time processing. Hyperspectral scenes from real missions are used as our testing data. In previous studies, with$k^{2}$-raster, the size of the hyperspectral data was reduced by up to 52% compared with the uncompressed data. In this letter, we continue to explore novel ways of further reducing the data size and access time. First, we examine the possibility of using the raster matrix of hyperspectral data without any padding (unpadded matrix) while still being able to compress the structure and access the data. Second, we examine some integer encoders, more specifically the Simple family. We discuss their ability to provide random element access and compare them with directly addressable codes (DACs), the integer encoder used in the original description for$k^{2}$-raster. Experiments show that the use of unpadded matrices has improved the storage size up to 6% while the use of a different integer encoder reduces the storage size up to 6% and element access time up to 20%.

On-board multi- and hyperspectral instruments acquire large volumes of data that need to be processed with the limited computational and storage resources. In this context, the CCSDS 123.0-B-2 standard emerges as an interesting option to compress multi- and hyperspectral images on-board satellites, supporting both lossless and near-lossless compression with low complexity and reduced power consumption. Nonetheless, the inclusion of a feedback loop in the CCSDS 123.0-B-2 predictor to support near-lossless compression introduces significant data dependencies that hinder real-time processing, particularly due to the presence of a quantization stage within this loop. This work provides an analysis of the aforementioned data dependencies and proposes two strategies aiming at maximizing throughput in hardware implementations and thus enabling real-time processing. In particular, through an elaborate mathematical derivation, the quantization stage is removed completely from the feedback loop. This reduces the critical path, which allows for shorter initiation intervals in a pipelined hardware implementation and higher throughput. This is achieved without any impact in the compression performance, which is identical to the one obtained by the original data flow of the predictor.

The increase in popularity of point-cloud-oriented applications has triggered the development of specialized compression algorithms. In this paper, a novel algorithm is developed for the lossless geometry compression of voxelized point clouds following an intra-frame design. The encoded voxels are arranged into runs and are encoded through a single-pass application directly on the voxel domain. This is done without representing the point cloud via an octree nor rendering the voxel space through an occupancy matrix, therefore decreasing the memory requirements of the method. Each run is compressed using a context-adaptive arithmetic encoder yielding state-of-the-art compression results, with gains of up to 15% over TMC13, MPEG's standard for point cloud geometry compression. Several proposed contributions accelerate the calculations of each run's probability limits prior to arithmetic encoding. As a result, the encoder attains a low computational complexity described by a linear relation to the number of occupied voxels leading to an average speedup of 1.8 over TMC13 in encoding speeds. Various experiments are conducted assessing the proposed algorithm's state-of-the-art performance in terms of compression ratio and encoding speeds.

Point cloud data have been put under the spotlight by many applications that play an increasingly important role in our every day lives. Their large size and ever-growing prevalent use cases have raised the interest in specialized compression algorithms for point cloud data. In this paper we propose a lossless intra-frame encoder for point cloud geometry. It relies on a single projection of the entire point cloud on a predetermined plane, combined with a context-adaptive binary arithmetic encoder. Our approach simplifies the current best performing approach for intra-frame compression. The experimental results indicate that our proposal not only improves the performance of all other intra-frame approaches, but it even surpasses the performance of state-of-the-art inter-frame approaches. Furthermore, we suggest to replace the adaptive encoder with a semi-adaptive approach for further performance gains.

Biorthogonal graph wavelet filterbanks, also known as GraphBior, are one of the most popular graph transforms used in image compression, but up to now, they could be designed based on two known admissible fundamental matrices: i) the random walk Laplacian, which heavily penalizes low degree pixels, and ii) the normalized Laplacian, which lacks a zero-DC response. By exploiting a new extension of the admissibility condition in GraphBior we propose a new fundamental matrix with the goal of distributing the errors of GraphBior more uniformly across pixels with different node degrees. Furthermore the proposed matrix preserves high energy compaction linked to the zero-DC GraphBior variation.

Nowadays entropy encoders are part of almost all data compression methods, with the Asymmetrical Numeral Systems (ANS) family of entropy encoders having recently risen in popularity. Entropy encoders based on thetabledvariant of ANS are known to provide varying performances depending on their internal design. In this paper, we present a method that calculates encoder redundancies in almost linear time, which translates in practice to thousand-fold speedups in redundancy calculations for small automatons, and allows redundancy calculations for automatons with tens of millions of states that would be otherwise prohibitive. We also address the problem of improving tabled ANS encoder designs, by employing the aforementioned redundancy calculation method in conjunction with a stochastic hill climbing strategy. The proposed approach consistently outperforms state-of-the-art methods in tabled ANS encoder design. For automatons of twice the alphabet size, experimental results show redundancy reductions around 10% over the default initialization method and over 30% for random initialization.

The use of whole-slide images (WSIs) in pathology entails stringent storage and transmission requirements because of their huge dimensions. Therefore, image compression is an essential tool to enable efficient access to these data. In particular, color transforms are needed to exploit the very high degree of inter-component correlation and obtain competitive compression performance. Even though the state-of-the-art color transforms remove some redundancy, they disregard important details of the compression algorithm applied after the transform. Therefore, their coding performance is not optimal. We propose an optimization method called mosaic optimization for designing irreversible and reversible color transforms simultaneously optimized for any given WSI and the subsequent compression algorithm. Mosaic optimization is designed to attain reasonable computational complexity and enable continuous scanner operation. Exhaustive experimental results indicate that, for JPEG 2000 at identical compression ratios, the optimized transforms yield images more similar to the original than the other state-of-the-art transforms. Specifically, irreversible optimized transforms outperform the Karhunen-Loève Transform in terms of PSNR (up to 1.1 dB), the HDR-VDP-2 visual distortion metric (up to 3.8 dB), and the accuracy of computer-aided nuclei detection tasks (F1 score up to 0.04 higher). In addition, reversible optimized transforms achieve PSNR, HDR-VDP-2, and nuclei detection accuracy gains of up to 0.9 dB, 7.1 dB, and 0.025, respectively, when compared with the reversible color transform in lossy-to-lossless compression regimes.

Digital cameras have become ubiquitous for amateur and professional applications. The raw images captured by digital sensors typically take the form of color filter array (CFA) mosaic images which must be “developed” (via digital signal processing) before they can be viewed. Photographers and scientists often repeat the “development process” using different parameters to obtain images suitable for different purposes. Since the development process is generally not invertible it is commonly desirable to store the raw (or undeveloped) mosaic images indefinitely. Uncompressed mosaic image file sizes can be more than 30 times larger than those of developed images stored in JPEG format. Thus data compression is of interest. Several compression methods for mosaic images have been proposed in the literature. However they all require a custom decompressor followed by development-specific software to generate a displayable image. In this paper a novel compression pipeline that removes these requirements is proposed. Specifically mosaic images can be losslessly recovered from the resulting compressed files and more significantly images can be directly viewed (decompressed and developed) using only a JPEG 2000 compliant image viewer. Experiments reveal that the proposed pipeline attains excellent visual quality while providing compression performance competitive to that of state-of-the-art compression algorithms for mosaic images.

Hyperspectral imaging captures the spectral responses of different wavelengths per pixel for an entire image. Because the number of spectral bands is large, efficient compression of hyperspectral images is important. Leveraging on recent advances in graph signal processing (GSP), in this paper we propose to encode a hyperspectral image in groups of ω spectral bands using graph wavelets, exploiting correlations along both the spatial and the spectral dimensions. Specifically, along the spatial dimension, we estimate the inter-pixel correlations for all adjacent pixel pairs from the last image in the previous coded band group. Along the spectral dimension, we first divide an image into different spatial regions with similar spectral responses, and encode the spectral signature (correlations along the spectrum) for each region as side information (SI). The spatial / spectral correlations are used to compute edge weights to construct a graph for signal-adaptive graph wavelet based compression. Experimental results suggest that our proposal can outperform existing schemes noticeably at comparable complexity.

The Consultative Committee for Space Data Systems (CCSDS) has issued several data compression standards devised to reduce the amount of data transmitted from satellites to ground stations. This paper introduces a contextual arithmetic encoder for on-board data compression. The proposed arithmetic encoder checks the causal adjacent neighbors, at most, to form the context and uses only bitwise operations to estimate the related probabilities. As a result, the encoder consumes few computational resources, making it suitable for on-board operation. Our coding approach is based on the prediction and mapping stages of CCSDS-123 lossless compression standard, an optional quantizer stage to yield lossless or near-lossless compression and our proposed arithmetic encoder. For both lossless and near-lossless compression, the achieved coding performance is superior to that of CCSDS-123, M-CALIC, and JPEG-LS. Taking into account only the entropy encoders, fixed-length codeword is slightly better than MQ and interleaved entropy coding.

The coding and transmission of the massive datasets captured by Earth Observation (EO) satellites is a critical issue in current missions. The conventional approach is to use compression on board the satellite to reduce the size of the captured images. This strategy exploits spatial and/or spectral redundancy to achieve compression. Another type of redundancy found in such data is the temporal redundancy between images of the same area that are captured at different instants of time. This type of redundancy is commonly not exploited because the required data and computing power are not available on board the satellite. This paper introduces a coding scheme for EO satellites able to exploit this redundancy. Contrary to traditional approaches, the proposed scheme employs both the downlink and the uplink of the satellite. Its main insight is to compute and code the temporal redundancy on the ground and transmit it to the satellite via the uplink. The satellite then uses this information to compress more efficiently the captured image. Experimental results for Landsat 8 images indicate that the proposed dual link image coding scheme can achieve higher coding performance than traditional systems for both lossless and lossy regimes.

The Karhunen-Loeve Transform (KLT) is a popular transform used in multiple image processing scenarios. Sometimes, the application of the KLT is not carried out as a single transform over an entire image. Rather, the image is divided into smaller spatial regions (segments), each of which is transformed by a smaller dimensional KLT. Such a situation may penalize the transform efficiency. An improvement for the segmented KLT, aiming at mitigating discontinuities arising on the edge of adjacent regions, is proposed in this paper. In the case of moderately varying image regions, discontinuities occur as the consequence of disregarded similarity between transform domains, as the order and sign of eigenvectors in the transform matrices are mismatched. In the proposed method, the KLT is adjusted to guarantee the best achievable similarity via the optimal assignment and sign correspondence for eigenvectors. Experimental results indicate that the proposed transform improves the similarity between transform domains, and reduces RMSE on the edge of adjacent regions. In consequence, images processed by the adjusted KLT present better cohesion and continuity between independently transformed regions.

The analysis techniques applied to DNA microarray images are under active development. As new techniques become available, it will be useful to apply them to existing microarray images to obtain more accurate results. The compression of these images can be a useful tool to alleviate the costs associated to their storage and transmission. The recently proposed Relative Quantizer (RQ) coder provides the most competitive lossy compression ratios while introducing only acceptable changes in the images. However, images compressed with the RQ coder can only be reconstructed with a limited quality, determined before compression. In this work, a progressive lossy-to-lossless scheme is presented to solve this problem. First, the regular structure of the RQ intervals is exploited to define a lossy-to-lossless coding algorithm called the Progressive RQ (PRQ) coder. Second, an enhanced version that prioritizes a region of interest, called the PRQ-region of interest (ROI) coder, is described. Experiments indicate that the PRQ coder offers progressivity with lossless and lossy coding performance almost identical to the best techniques in the literature, none of which is progressive. In turn, the PRQ-ROI exhibits very similar lossless coding results with better rate-distortion performance than both the RQ and PRQ coders.

DNA microarrays are one of the fastest-growing new technologies in the field of genetic research, and DNA microarray images continue to grow in number and size. Since analysis techniques are under active and ongoing development, storage, transmission and sharing of DNA microarray images need be addressed, with compression playing a significant role. However, existing lossless coding algorithms yield only limited compression performance (compression ratios below 2:1), whereas lossy coding methods may introduce unacceptable distortions in the analysis process. This work introduces a novel Relative Quantizer (RQ), which employs non-uniform quantization intervals designed for improved compression while bounding the impact on the DNA microarray analysis. This quantizer constrains the maximum relative error introduced into quantized imagery, devoting higher precision to pixels critical to the analysis process. For suitable parameter choices, the resulting variations in the DNA microarray analysis are less than half of those inherent to the experimental variability. Experimental results reveal that appropriate analysis can still be performed for average compression ratios exceeding 4.5:1.

Recent years have seen the upraising of a new type of processors strongly relying on the Single Instruction, Multiple Data (SIMD) architectural principle. The main idea behind SIMD computing is to apply a flow of instructions to multiple pieces of data in parallel and synchronously. This permits the execution of thousands of operations in parallel, achieving higher computational performance than with traditional Multiple Instruction, Multiple Data (MIMD) architectures. The level of parallelism required in SIMD computing can only be achieved in image coding systems via microscopic parallel strategies that code multiple coefficients in parallel. Until now, the only way to achieve microscopic parallelism in bit plane coding engines was by executing multiple coding passes in parallel. Such a strategy does not suit well SIMD computing because each thread executes different instructions. This paper introduces the first bit plane coding engine devised for the fine grain of parallelism required in SIMD computing. Its main insight is to allow parallel coefficient processing in a coding pass. Experimental tests show coding performance results similar to those of JPEG2000.

This letter proposes a near-lossless coder for hyperspectral images. The coding technique is fully embedded and minimizes the distortion in the l2-norm initially and in the $l_\infty$-norm subsequently. Based on a two-stage near-lossless compression scheme, it includes a lossy and a near-lossless layer. The novelties are the observation of the convergence of the entropy of the residuals in the original domain and in the spectral-spatial transformed domain and an embedded near-lossless layer. These contributions enable a progressive transmission while optimizing both signal-to-noise ratio (SNR) and peak absolute error (PAE) performance. The embeddedness is accomplished by bit-plane encoding plus arithmetic encoding. Experimental results suggest that the proposed method yields a highly competitive coding performance for hyperspectral images, outperforming multicomponent JPEG2000 for the l∞-norm and pairing its performance for the l2-norm, and also outperforming M-CALIC in the near-lossless case, i.e., for PAE ≥ 5.

Spectral transforms are tools commonly employed in multi- and hyperspectral data compression to decorrelate images in the spectral domain. The pairwise orthogonal transform (POT) is one such transform that has been specifically devised for resource-constrained contexts similar to those found on board satellites or airborne sensors. Combining the POT with a 2-D coder yields an efficient compressor for multi- and hyperspectral data. However, a drawback of the original POT is that its dynamic range expansion, i.e., the increase in bit depth of transformed images, is not constant, which may cause problems with hardware implementations. Additionally, the dynamic range expansion is often too large to be compatible with the current 2-D standard CCSDS 122.0-B-1. This paper introduces the isorange POT, a derived transform that has a small and limited dynamic range expansion, compatible with CCSDS 122.0-B-1 in almost all scenarios. Experimental results suggest that the proposed transform achieves lossy coding performance close to that of the original transform. For lossless coding, the original POT and the proposed isorange POT achieve virtually the same performance.

Wavelet-based coding systems commonly employ uniform scalar deadzone quantization (USDQ) together with a bitplane coding strategy to progressively refine image quality. Our previous work presents a quantization scheme that employs 2 step sizes depending on the magnitude of the coefficients. This 2-step scalar deadzone quantization (2SDQ) scheme is introduced in the framework of JPEG2000 by modifying all coefficients within a codeblock to enhance the quality of the image while transmitting fewer bitplanes than those needed with a conventional USDQ scheme. This paper extends our prior work by applying the 2SDQ in a subblock level, i.e., in small sets of coefficients, called cells, selected within a codeblock. Combined with rate-distortion optimization techniques, the proposed cell-based 2SDQ can help to code high quality images employing even fewer bitplanes than those needed with our previous strategy. This may be especially useful for high-dynamic range images or for devices with constrained resources.

DNA micro arrays are useful to identify the function and regulation of a large number of genes in a single experiment, even whole genomes. In this work, we analyze the relationship between DNA micro array image histograms and the compression performance of loss less JPEG2000. Also, a reversible transform based on histogram swapping is proposed. Intensive experimental results using different coding parameters are discussed. Results suggest that this transform improves previous loss less JPEG2000 results on all DNA micro array image sets.

Embedded quantization is a mechanism employed by lossy image coding systems to successively refine the distortion of an image. Commonly, it is conducted through a uniform scalar dead zone quantizer (USDQ) together with a bitplane coding strategy (BPC). Although this scheme is convenient for current hardware architectures and achieves competitive coding performance, it establishes the embedded quantizer without allowing major variations. This paper studies the design of non-restricted embedded quantizers with the aim to determine a quantization scheme that provides (near-)optimal performance for the lossy compression of images at low rates. Results suggest that optimally designed quantization schemes can achieve slightly better performance than that of USDQ+BPC by employing a non-uniform quantizer that requires fewer quantization stages.

Hyperspectral data lossy compression has not yet achieved global acceptance in the remote sensing community, mainly because it is generally perceived that using compressed images may affect the results of posterior processing stages. This possible negative effect, however, has not been accurately characterized so far. In this letter, we quantify the impact of lossy compression on two standard approaches for hyperspectral data exploitation: spectral unmixing, and supervised classification using support vector machines. Our experimental assessment reveals that different stages of the linear spectral unmixing chain exhibit different sensitivities to lossy data compression. We have also observed that, for certain compression techniques, a higher compression ratio may lead to more accurate classification results. Even though these results may seem counterintuitive, this work explains these observations in light of the spatial regularization and/or whitening that most compression techniques perform and further provides recommendations on best practices when applying lossy compression prior to hyperspectral data classification and/or unmixing.

Spectral transforms are widely used for the codification of remote-sensing imagery, with the Karhunen-Loêve transform (KLT) and wavelets being the two most common transforms. The KLT presents a higher coding performance than the wavelets. However, it also carries several disadvantages: high computational cost and memory requirements, difficult implementation, and lack of scalability. In this paper, we introduce a novel transform based on the KLT, which, while obtaining a better coding performance than the wavelets, does not have the mentioned disadvantages of the KLT. Due to its very small amount of side information, the transform can be applied in a line-based scheme, which particularly reduces the transform memory requirements. Extensive experimental results are conducted for the Airborne Visible/Infrared Imaging Spectrometer and Hyperion images, both for lossy and lossless and in combination with various hyperspectral coders. The results of the effects on Reed Xiaoli anomaly detection and k-means clustering are also included. The theoretical and experimental evidences suggest that the proposed transform might be a good replacement for the wavelets as a spectral decorrelator in many of the situations where the KLT is not a suitable option.

Context-adaptive binary arithmetic coding (CABAC) is a popular approach to diminish the statistical redundancy of symbols emitted by bitplane image coders. The main idea behind CABAC is to set up appropriate context models for coefficients, and to adapt probability estimates for each context to the nonstationary statistical behavior of symbols as more data are coded. This works introduces a mathematical model to determine probability estimates conceived from a characterization of the signal's nature within wavelet subbands. The proposed model assumes stationary statistical behavior for emitted symbols, thus the context-adaptive process carried out by CABAC is avoided. Experimental results in the framework of JPEG2000 suggest 2% increment on coding efficiency.

The Karhunen-Loêve transform (KLT) is widely used in hyperspectral image compression because of its high spectral decorrelation properties. However, its use entails a very high computational cost. To overcome this computational cost and to increase its scalability, in this paper, we introduce a multilevel clustering approach for the KLT. As the set of different multilevel clustering structures is very large, a two-stage process is used to carefully pick the best members for each specific situation. First, several candidate structures are generated through local search and eigenthresholding methods, and then, candidates are further screened to select the best clustering configuration. Two multilevel clustering combinations are proposed for hyperspectral image compression: one with the coding performance of the KLT but with much lower computational requirements and increased scalability and another one that outperforms a lossy wavelet transform, as spectral decorrelator, in quality, cost, and scalability. Extensive experimental validation is performed, with images from both the AVIRIS and Hyperion sets, and with JPEG2000, 3D-TCE, and CCSDS-Image Data Compression recommendation as image coders. Experiments also include classification-based results produced by k-means clustering and Reed-Xiaoli anomaly detection.

The RKLT is a lossless approximation to the KLT, and has been recently employed for progressive lossy-to-lossless coding of hyperspectral images. Both yield very good coding performance results, but at a high computational price. In this paper we investigate two RKLT clustering approaches to lessen the computational complexity problem: a normal clustering approach, which still yields good performance; and a multi-level clustering approach, which has almost no quality penalty as compared to the original RKLT. Analysis of rate-distortion evolution and of lossless compression ratio is provided. The proposed approaches supply additional benefits, such as spectral scalability, and a decrease of the side information needed to invert the transform. Furthermore,since with a clustering approach, SERM factorization coefficients are bounded to a finite range, the proposed methods allow coding of large three dimensional images within JPEG2000.

Progressive lossy-to-lossless methods for hyper-spectral image coding are becoming common in remote-sensing. However, as remote-sensing imagery is sometimes fed directly into an automated process, there are several alternative distortion measures directed to quantify the image quality with regard to how this process will perform. In this scenario, we investigate the quality evolution in the lossy regime of progressive lossy-to-lossless and perform a detailed evaluation.