Suffix-based data structures are widely used for pattern recognition. In one dimension, there exist suffix automata and suffix trees, whereas in two dimensions only suffix trees are known [1]. A two-dimensional suffix at$(i, j)$is the largest square submatrix starting there and may not be unique. We introduce the CDAIG, an automaton that recognizes such two-dimensional suffixes analogously to the Isuffix Tree [1] and is built by adapting [2]. We evaluated the complexity of the CDAIG by aligning 200 randomly selected 1000 bp-long coding chunks from the Escherichia coli genome with homologs from Enterobacteriaceae species identified by BLAST [3]. Each alignment was truncated to 750 bp, all differences (substitutions and gaps) were counted, and a CDAIG was built for each pair. Figure 1 shows that edge and suffix counts increase with the number of differences, capturing the rise in sequence divergence and reflecting the phylogenetic distance between E. coli and the compared species. Datasets are available at [E. coli dataset], and the full implementation at [CDAIG data structure].

High-throughput crystallography experiments at synchrotron facilities generate massive volumes of raw diffraction data, challenging storage and long-term data management. The Hierarchical Data Format version 5 (HDF5) is widely used to organize these datasets, but its native compressors often provide limited compression ratios. In this study, we systematically evaluate a broad range of lossless compression algorithms on rotational macromolecular crystallography (MX) and serial synchrotron crystallography (SSX) datasets, including HDF5-native coders. Our results show that preprocessing strategies significantly influence compression performance, with some codecs benefiting from bit- or Byteshuffling, while others perform optimally without it. JPEG XL and the algorithm described in [1] achieve the highest weighted average compression ratios across all datasets, outperforming the best HDF5-native methods. These findings provide a benchmark for selecting optimal lossless compression strategies for large-scale crystallographic datasets.

In this work, lossless compression algorithms are evaluated on a variety of real, current as-tronomical images. The test dataset comprises raw (integer) and processed (floating-point) images of discrete and extensive astronomical objects, captured by spatial or terrestrial tele-scopes. Compression techniques herein analyzed are chosen to be representative of the most recent algorithms devised for astronomical data, as well as the most commonly employed compressors employed in real observatories. Experimental results suggest that coding techniques such as RICE and HCOMPRESS, typically employed in world-class observatories such as Roque de los Muchachos, do not produce the best possible lossless compression results. Instead, JPEG-LS, LZMA and NDZIP yield the best compression ratio results for 16-bit data (2.72), floating-point data (2.38) and radio data (1.81), respectively. Therefore, the efficiency with which data are stored and transmitted by these observatories could be significantly improved by selectively employing the aforementioned algorithms.

Hyperspectral images are typically highly correlated along their spectrum, and this similarity is usually found to cluster in intervals of consecutive bands. We identified 5 such intervals in AVIRIS uncalibrated data (i.e., as captured on-board). These 5 intervals maximised the average spectral correlation along the 224 band spectrum. The resulting in-tervals were composed of bands 1–40, 41–96, 97–155, 156–165, and 166–224, as seen in the figure to the right.

Methods based on machine learning (ML) have been recently proposed to improve upon traditional block-based intra-prediction algorithms in modern video codecs [1,2]. Their performance, however, depends on the amount, quality and relevance of the training data. Furthermore, they require signaling the learned parameters to the decoder, thus increasing compressed data volumes. In this work, six new prediction modes based on fully-connected neural networks (FC-NNs) are proposed that avoid the two aforementioned shortcomings. To do so, 1-layer FC-NNs are used, whose parameters are rened by overfitting on the data samples being predicted. This allows to replicate the parameter optimization process at the decoder under a lossless compression regime without requiring any additional side information. Each proposed ML-based mode is based on a 1-layer FC-NN that predicts a block of size k k in a column-wise or row-wise manner using as input a subset of the reference samples used by traditional intra-prediction. Each subset, which varies for each column or row, is computed by averaging a number of reference samples to reduce noise [3]. Experimental results based on several video frames indicate that the proposed ML-based modes are selected as the best modes between 11% and 93% of the times. (see Table 1). When used in a hybrid intra-prediction framework that also includes HEVC's modes, the proposed ML-based modes increase prediction accuracy by between 0.56 dB and 7.01 dB PSNR, with respect to using only HEVC's modes.

Point cloud data have been put under the spotlight by many applications that play an increasingly important role in our every day lives. Their large size and ever-growing prevalent use cases have raised the interest in specialized compression algorithms for point cloud data. In this paper we propose a lossless intra-frame encoder for point cloud geometry. It relies on a single projection of the entire point cloud on a predetermined plane, combined with a context-adaptive binary arithmetic encoder. Our approach simplifies the current best performing approach for intra-frame compression. The experimental results indicate that our proposal not only improves the performance of all other intra-frame approaches, but it even surpasses the performance of state-of-the-art inter-frame approaches. Furthermore, we suggest to replace the adaptive encoder with a semi-adaptive approach for further performance gains.

Marlin [1, 2] is a Variable-to-Fixed (VF) codec optimized for high decoding speed through the use of small sized dictionaries that fit in the L1 cache of most CPUs. While the size of Marlin dictionaries is adequate for decoding, they are still too large to be encoded fast. We address this problem by proposing two techniques to reduce the alphabet size. The first technique is to encode rare symbols in their own segment, and the second is to combine Marlin dictionaries with Rice encoding, hence our name Rice-Marlin for our new codec. Using those techniques [3], we are able to reduce the size of Marlin dictionaries by a factor of 16, not only enabling faster encoding speed, but also achieving better compression efficiency.

Marlin [1] is a Variable-to-Fixed (VF) codec optimized for decoding speed. To achieve its speed, Marlin does not encode the current state of the input source, penalyzing compression ratio. In this paper we address this penalty by partially encoding the current state of the input in the lower bits of the codeword. Those bits select which chapter in the dictionary must be used to decode the next codeword. Each chapter is specialized for a subset of states, improving compression ratio. At the same time, we use one victim chapter to encode all rare symbols, increasing the efficiency of the rest of them. The decoding algorithm remains the same, only now codewords have overlapping bits. Mapping techniques allow us to combine common chapters and thus keep an efficient use of the L1 cache. We evaluate our approach with both synthetic and real data sets, and show significant improvements in low entropy sources, where compression efficiency can improve from 93.9% to 98.6%.

Fast and efficient coding techniques are being increasingly required to meet the complexity restrictions of on-board satellite compression. The recently proposed Regression Wavelet Analysis (RWA) has proven to be highly effective as a spectral transform for coding remote sensing images. The algorithm is based on a pyramidal prediction, using multiple regression analysis, to tackle residual data dependencies in the wavelet domain. RWA combines low complexity and reversibility and has demonstrated competitive performance for lossless and progressive lossy-to-lossless compression superior to the state-of-the-art predictive-based CCSDS-123.0 and the widely used transform-based principal component analysis (PCA). In this paper we introduce a very low-complexity RWA approach, where prediction is based on only a few components, while the performance is maintained. When RWA computational complexity is taken to an extremely low level, careful model selection is necessary. Contrary to expensive selection procedures, we propose a simple and efficient strategy called neighbor selection for using small regression models. On a set of well-known and representative hyperspectral images, these small models maintain the excellent coding performance of RWA, while reducing the computational cost by about 90%.

We present Marlin, a variable-to-fixed (VF) codec optimized for decoding speed. Marlin builds upon a novel way of constructing VF dictionaries that maximizes efficiency for a given dictionary size. On a lossless image coding experiment, Marlin achieves a compression ratio of 1.94 at 2494MiB/s. Marlin is as fast as state-of-the-art high-throughput codecs (e.g., Snappy, 1.24 at 2643MiB/s), and its compression ratio is close to the best entropy codecs (e.g., FiniteStateEntropy, 2.06 at 523MiB/s). Therefore, Marlin enables efficient and high throughput encoding for memoryless sources, which was not possible until now.

Regression Wavelet Analysis (RWA) is a novel wavelet-based scheme for coding hyperspectral images that employs multiple regression analysis to exploit the relationships among spectral wavelet-transformed components. The scheme is based on a pyramidal prediction, using different regression models, to increase the statistical independence in the wavelet domain. For lossless coding, RWA has proven to be superior to other spectral transform like PCA and to the best and most recent coding standard in remote sensing, CCSDS-123.0. In this paper we show that RWA also allows progressive lossy-to-lossless (PLL) coding and that it attains a rate-distortion performance superior to those obtained with state-of-the-art schemes. To take into account the predictive significance of the spectral components, we propose a Prediction Weighting scheme for JPEG2000 that captures the contribution of each transformed component to the prediction process.

The coding and transmission of the massive datasets captured by Earth Observation (EO) satellites is a critical issue in current missions. The conventional approach is to use compression on board the satellite to reduce the size of the captured images. This strategy exploits spatial and/or spectral redundancy to achieve compression. Another type of redundancy found in such data is the temporal redundancy between images of the same area that are captured at different instants of time. This type of redundancy is commonly not exploited because the required data and computing power are not available on board the satellite. This paper introduces a coding scheme for EO satellites able to exploit this redundancy. Contrary to traditional approaches, the proposed scheme employs both the downlink and the uplink of the satellite. Its main insight is to compute and code the temporal redundancy on the ground and transmit it to the satellite via the uplink. The satellite then uses this information to compress more efficiently the captured image. Experimental results for Landsat 8 images indicate that the proposed dual link image coding scheme can achieve higher coding performance than traditional systems for both lossless and lossy regimes.

Whole-slide images (WSIs) are high-resolution, 2D, color digital images that are becoming valuable tools for pathologists in clinical, research and formative scenarios. However, their massive size is hindering their widespread adoption. Even though lossy compression can effectively reduce compressed file sizes without affecting subsequent diagnoses, no lossy coding scheme tailored for WSIs has been described in the literature. In this paper, a novel strategy called OptimizeMCT is proposed to increase the lossy coding performance for this type of images. In particular, an optimization method is designed to find image-specific multi-component transforms (MCTs) that exploit the high inter-component correlation present in WSIs. Experimental evidence indicates that the transforms yielded by OptimizeMCT consistently attain better coding performance than the Karhunen-Loève Transform (KLT) for all tested lymphatic, pancreatic and renal WSIs. More specifically, images reconstructed at the same bitrate exhibit average PSNR values 2.85~dB higher for OptimizeMCT than for the KLT, with differences of up to 5.17 dB.

Wavelet-based coding systems commonly employ uniform scalar deadzone quantization (USDQ) together with a bitplane coding strategy to progressively refine image quality. Our previous work presents a quantization scheme that employs 2 step sizes depending on the magnitude of the coefficients. This 2-step scalar deadzone quantization (2SDQ) scheme is introduced in the framework of JPEG2000 by modifying all coefficients within a codeblock to enhance the quality of the image while transmitting fewer bitplanes than those needed with a conventional USDQ scheme. This paper extends our prior work by applying the 2SDQ in a subblock level, i.e., in small sets of coefficients, called cells, selected within a codeblock. Combined with rate-distortion optimization techniques, the proposed cell-based 2SDQ can help to code high quality images employing even fewer bitplanes than those needed with our previous strategy. This may be especially useful for high-dynamic range images or for devices with constrained resources.

This work defines an entropy-based measure aimed to establish the compression limits of wavelet-based image coding. This measure serves to appraise the efficiency of current codecs, determining whether there is margin for their improvement or not. Also, it may help to design new compression schemes that target a particular type of images and/or compression rates.

This works focuses on the High Efficiency Video Coding (HEVC) standard as a compression method to be potentially adopted by the Digital Imaging and Communications in Medicine (DICOM) standard. We are particularly interested in improving the lossless compression efficiency of the intra coding process for grayscale anatomical medical images. We focus on intra coding due to its low complexity and outstanding compression results, as well as the fact that it allows coding high-dimensional medical images on a slice-by-slice basis. This is especially advantageous for cases when only a small set of slices needs to be accessed without the need to decode the entire data set. Based on the characteristics of grayscale anatomical medical images, specifically their large amount of edge information and frequent number of patterns depicted on various directions, we propose improvements to HEVC intra coding based on sample-by-sample (SbS) differential pulse code modulation (DPCM) with equal displacements so the density of prediction modes is constant in all directions. Performance evaluations over MRI, CT and X-ray angiography sequences show that the proposed improvements outperform current HEVC lossless intra coding, achieving average coding gains of 6%.

DNA micro arrays are state-of-the-art tools in biological and medical research. In this work, we discuss the suitability of lossy compression for DNA micro array images and highlight the necessity for a distortion metric to assess the loss of relevant information. We also propose one possible metric that considers the basic image features employed by most DNA micro array analysis techniques. Experimental results indicate that the proposed metric can identify and differentiate important and unimportant changes in DNA micro array images.

Visually lossless coding is a method through which an image is coded with numerical losses that are not noticeable by visual inspection. Contrary to numerically lossless coding, visually lossless coding can achieve high compression ratios. In general, visually lossless coding is approached from the point of view of the encoder, i.e., as a procedure devised to generate a compressed code stream from an original image. If an image has already been encoded to a very high fidelity (higher than visually lossless - perhaps even numerically lossless), it is not straightforward to create a just visually lossless version without fully re-encoding the image. However, for large repositories, re-encoding may not be a suitable option. A visually lossless decoder might be useful to decode, or to parse and transmit, only the data needed for visually lossless reconstruction. This work introduces a decoder for JPEG 2000 code streams that identifies and decodes the minimum amount of information needed to produce a visually lossless image. The main insights behind the proposed method are to estimate the variance of the code blocks before the decoding procedure, and to determine the visibility thresholds employing a well-known model from the literature. The main advantages are faster decoding and the possibility to transmit visually lossless images employing minimal bit rates.

Computed Tomography (CT) devices irradiate a (human) body with controlled amounts of X-ray to produce an image where different substance (lung, tissue, vessels, etc.) can be identified unequivocally. Commonly, CT devices also capture areas that do not belong to the human body. Such areas are referred to as air pixels, and may contain imaging artifacts. The air pixels are irrelevant for the medical diagnostic and provoke an important degradation in coding efficiency. In order to improve coding performance, we propose an air filtering technique based on a thresholding in the wavelet domain. The thresholds are determined through the existing relation between wavelet coefficients and image samples, which can be expressed in terms of a probability function. The proposed scheme filters air pixels in the wavelet domain by removing coefficients that are below a given threshold. The thresholds are estimated for different resolution levels and subbands, obtaining a probability of 70% to correctly filter air pixels. Although the proposed technique introduces an slight distortion in terms of RMSE in the biological area, this distortion is negligible compared with the state-of-the-art HDCS filter. These results suggest that the rate-distortion coding performance of our proposal and HDCS outperform significantly the coding performance of JPEG2000. In addition, Table 1 provides the RMSE of the HDCS and our proposal when compared with the original image, indicating that our proposal introduces much less RMSE distortion.

Summary form only given. X-ray angiographic (angio) images are widely used for identifying irregularities in the vascular system. Because of their high spatial resolution and the increasingly amount of X-ray angio images generated, compression of these images is becoming increasingly appealing. In this paper, we introduce a diagnostically lossless compression scheme for X-ray angio images. The coding scheme relies on a novel method based on ray casting and a-shapes for distinguishing the clinically relevant Region of Interest from the background. The background is then suppressed to increase data redundancy, allowing to achieve a higher coding performance. Experimental results suggest that the proposed scheme correctly identifies the Region of Interest in X-ray angio images and achieves more than 2 bits per pixel reduction in average as compared to the case of compression with no background suppression. Results are reported here for 20 out of 25 images compressed using various lossless compression methods.

Preclinical imaging is a key enabling technology for medical research, such as in drug discovery, cancer detection, or osteoporosis screening. Europe is investing a significant amount of resources in a distributed phenotype study, carried out using Micro Computed Tomography (CT) images, which are acquired at very high resolutions to ease the detection of skeleton malformations. Unfortunately, like other CT images, MicroCT images commonly contain a notable amount of noise emitted by the acquisition device, which hinders the encoding of such images. In order to improve coding performance, we propose to use the Hounsfield Scale to establish a relationship between CT values and biological tissue together with an air-filtering approach, which modifies only samples located outside the biological area without penalizing the usefulness of the images to clinical experts, nor the visual perception of images. The proposed filter sets all samples having value lower than a specific threshold T to a constant figure.

DNA micro arrays are useful to identify the function and regulation of a large number of genes in a single experiment, even whole genomes. In this work, we analyze the relationship between DNA micro array image histograms and the compression performance of loss less JPEG2000. Also, a reversible transform based on histogram swapping is proposed. Intensive experimental results using different coding parameters are discussed. Results suggest that this transform improves previous loss less JPEG2000 results on all DNA micro array image sets.

Embedded quantization is a mechanism employed by lossy image coding systems to successively refine the distortion of an image. Commonly, it is conducted through a uniform scalar dead zone quantizer (USDQ) together with a bitplane coding strategy (BPC). Although this scheme is convenient for current hardware architectures and achieves competitive coding performance, it establishes the embedded quantizer without allowing major variations. This paper studies the design of non-restricted embedded quantizers with the aim to determine a quantization scheme that provides (near-)optimal performance for the lossy compression of images at low rates. Results suggest that optimally designed quantization schemes can achieve slightly better performance than that of USDQ+BPC by employing a non-uniform quantizer that requires fewer quantization stages.

The efficient transmission of large resolution images is a key aspect in many applications to minimize the transmission costs and to enhance the browsing experience. Among the currently available standards for the coding and transmission of imagery, JPEG2000 excels for its superior coding performance and advanced capabilities. The JPEG2000 Interactive Protocol (JPIP) minimizes the amount of information transmitted in a client-server scenario. Nonetheless, JPIP does not provide mechanisms to re-use data already delivered to clients browsing the same image within a local network. Common HTTP proxy servers are not able to understand the syntax of JPIP, thus specialized JPIP proxy servers are put in practice. This work improves the capabilities of traditional JPIP proxy servers by means of a user-navigation model that, together with prefetching strategies, allows the server to anticipate (potential) future requests of clients. Experimental evidence indicates that the introduction of the navigational model into a JPIP proxy server enhances the browsing experience notably.

Rate allocation methods are used for variable bit rate (VBR) video transmission. The Fast rate allocation through STepest descent (FAST) is a rate allocation method that achieves valid solutions in scenarios where buffer and bandwidth may vary from client to client, respecting buffer limits and fulfilling real time processing requeriments in a JPEG2000 framework. This work restates the optimization problem and implements an expansion for FAST algorithm, allowing the adaptation to possible changes in channel bandwidth while video still transferring. The proposed method considers buffer's fullness, bandwidth changes and number of frames remainig to transfer in order to provide a valid solution. Experimental results suggest the validity of solutions whereas maintaining the initial version features.

Radiation exposure increases the risk of inducing cancer in a patient when Computed Tomography is performed. Radiologists reduce the radiation dose to minimize the risk of cancer, consequently, noise is introduced in the image, considerably penalizing its quality. To enhance the quality of the image, different noise filters are developed. The filtered noise can be discarded due to its insignificance for the medical diagnosis. In this research, a coding scheme for Computed Tomography with JPEG2000 including a noise filtering stage is proposed. The JPEG2000 standard is selected in our coding approach because: it is supported in DICOM, the communication protocol in the medical setting, it includes progressive lossy-to-lossless coding, and it incorporates JPIP, the interactive transmission protocol. Extensive experimental tests using different images suggest that noise filtering does not penalize the visual quality, but allows substantial improvements in coding performance.

Remote browsing of images is receiving much attention lately, mostly in niche applications like geographical information systems or in the telemedicine scenery. Interactive transmission of compressed images has been identified as the most competitive approach, being JPIP, JPEG2000 Interactive Protocol, a key-enabler for these situations. Also, it has been reported that JPIP Proxy Servers help to increase the transmission performance, and that prefetching strategies help to lower responsiveness time. In this paper we contribute a smart JPIP Proxy Server that, thanks to a prefetching strategy undertaken during idle transmission times, largely improves the viewing experience of the final user because of its anticipation of future navigation requests. Experimental results are reported for a remote sensing and a medical environment, respectively performing panning and zoom in, showing enhanced performance in both cases.

The RKLT is a lossless approximation to the KLT, and has been recently employed for progressive lossy-to-lossless coding of hyperspectral images. Both yield very good coding performance results, but at a high computational price. In this paper we investigate two RKLT clustering approaches to lessen the computational complexity problem: a normal clustering approach, which still yields good performance; and a multi-level clustering approach, which has almost no quality penalty as compared to the original RKLT. Analysis of rate-distortion evolution and of lossless compression ratio is provided. The proposed approaches supply additional benefits, such as spectral scalability, and a decrease of the side information needed to invert the transform. Furthermore,since with a clustering approach, SERM factorization coefficients are bounded to a finite range, the proposed methods allow coding of large three dimensional images within JPEG2000.

Distortion estimation techniques are often employed in bitplane coding engines to minimize the computational load, or the memory requirements, of the encoder. A common approach is to determine distortion estimators that approximate the mean squared error decreases when data are successively coded and transmitted. Such estimators usually assume that coefficients are uniformly distributed in the quantization interval. Even though this assumption simplifies estimation, it does not exactly correspond with the nature of the signal. This work introduces new distortion estimators determined through a precise approximation of the coefficient's distribution within the quantization intervals. Experimental results obtained when our estimators are used for the post-compression rate-distortion optimization process of JPEG2000 suggest that they are able to approximate distortion with very high accuracy.

Region Of Interest (ROI) coding is a mechanism deployed in several image coding systems to enable different degrees of coding priority to specific regions of the image. JPEG2000 standard provides two ROI coding methods. However, both of them are based in mechanisms that scale the quantized coefficients. This compels to encode the additional bit-planes needed for the scaling, causing a penalization in the overall coding performance, and the ROIs can not be modified without a complete re-encoding of the image. This paper introduces two ROI coding methods that use Rate-distortion optimization techniques toprioritize arbitrary ROIs over the rest of the image without penalizing the overall coding performance. Experimental results suggest that the proposed methods achieve a close tooptimal accuracy, improving the Implicit ROI coding method in terms of ROI rate distortion performance.

In this paper a modification on the file syntax of the CCSDS Recommendation for Image Data Coding (CCSDS-122-B-1) is presented. With respect to the Recommendation, the proposed modification provides, among others, scalability by quality, spatial location, resolution, and component, and also allows multicomponent data coding. In addition, experimental results show that our proposal produces a meaningful improvement over the performance of the Recommendation. Reported results for hyperspectral data are alsocompetitive with JPEG2000, the reference technique.

Summary form only given. We adapted shape-adaptive coding and the BISK algorithm to new sign and refinement encoders, with the novelty of encoding separately the refinement bits for each set of coefficients having the same prefix. This allows the algorithm to capitalize the refinement redundancy among each of these sets. The proposed sign and refinement encoders may be considered independently and may be integrated to other bit plane encoders. However, the search scheme suggests a new complete bit plane encoder defined by an exhaustive two-valued shape-adaptive search (ETSE). Even though ETSE does not include arithmetic coding, the coding performance of ETSE is competitive when compared to other wavelet-based encoders which include arithmetic coding. Furthermore, as BISK, ETSE may perform compression of images with non-regular boundary

The JPEG2000 standard does not specify how to perform the rate control strategy, needed either to achieve a target bitrate, or to construct quality layers. In this paper, an efficient rate control algorithm is described. It is based on the interleaving of coding passes and it has a low computational complexity. The proposed algorithm encodes only the coding passes included in the final codestream, and it fully avoids the need of any post compression rate distortion stage. Extensive experimental results show that the encoding performance of our method is competitive and similar to the optimal strategy. In addition, this proposal allows the extraction of a target bitrate from a codestream without the need of either knowing the original image, or of decoding any part of the codestream. The performance of this kind of extraction is equivalent to that obtained when decompressing a codestream organized in quality layers.