HPC applications generate massive amounts of data that impose significant burdens on both storage and I/O systems. Although lossy compressors have been widely adopted in this scenario to reduce data volume, the SOTA approach fails to fully exploit redundancy because it relies on separate techniques that operate at incompatible granularities. Their suboptimal compression ratios leave I/O as the dominant bottleneck in data dumps/loads. Therefore, we propose MAC, a compression framework built upon existing SZ compressor. It leverages the alignment between HPC system characteristics and modeling-based arithmetic coding to balance the compression ratio improvement and time cost. Instead of applying Huffman coding and dictionary-based compressors like zstd or gzip sequentially on quantization factors as SZ does, MAC replaces them with an adaptive arithmetic encoder. Specifically, MAC first employs bit-packing on incoming quantization factors to reduce overhead, as these factors are typically small enough that standard 4-byte storage would impede processing efficiency. The system then constructs context with the knowledge of the length of each quantization factor, utilizing hash tables to store and retrieve historical occurrences. By leveraging two models with distinct prefix-matching strategies and integrating them via a logistic mixer, MAC yields substantial compression gains. This architecture ensures that compression and decompression latencies remain low enough to accelerate overall dump and load operations. Experiments show that MACSZ achieves a compression ratio improvement of over 25%, which translates directly into enhanced throughput on HPC cluster architectures as Fig 1 and 2 demonstrate.

The explosive growth of global data has created an increasing demand for archival storage, where efficient compression is crucial to reduce capacity cost. However, existing archival compressors face a fundamental tradeoff: mainstream methods (e.g., ZSTD with level$21 / 22$) offer limited compression ratios, while context-mixing compressors (e.g., LPAQ) achieve higher ratios but are often too slow for practical use. Therefore, we present CuCM, a GPU-powered context-mixing compressor to overcome this tradeoff. By introducing pre-learning and batch update mechanisms, CuCM resolves the data dependencies inherent in the autoregressive modeling of contextmixing compressors. During compression, CuCM processes each predefined vector as a single unit. It utilizes the current model to predict the probability distribution for the entire vector, deferring model updates until the vector is fully processed. During decompression, CuCM employs an aggressive look-ahead strategy, preassuming bit values for context construction. It then retains only the outcomes of hypotheses that remain consistent with the actual decoded data. Experiments like figure 1 show that CuCM achieves up to$12.6 \times$higher throughput than LPAQ while maintaining comparable compression ratios across both general-purpose and archival datasets.

Current compressors for OLTP databases perform well on text but face challenges with integers, although integers are a critical component of the workload. Most existing integer compressors are ineffective as a complementary solution, since they compress integers together and cannot decompress a certain integer individually, making them incompatible with the data access requirement of OLTP databases. To this end, we propose Apic, a precomputation-based arithmetic coding to efficiently compress each integers (a very tiny unit), though small data are always hard to compress, and ensure compatibility with OLTP datasets. Specifically, Apic presents Bitwidth-aware Precomputed Frequency and Prefixaware Precomputed Decoding to tackle challenges of applying arithmetic coding in this scenario, such as the substantial space costs of symbol frequencies and decompression complexity. Evaluations on real-world and desensitized commercial datasets suggest that Apic improves the compression ratio by up to 80% on integers over VByte, while preserving comparable decompression speed and thus query performance.

The row-oriented store model is the cornerstone component of modern online transaction processing (OLTP) database systems. In response to the massive increase in data within database systems, compression techniques are employed to enhance storage efficiency. Regrettably, current compression methods suffer from either the amplification issue due to coarse compression granularity or inefficient decompression operations, thus usually decreasing the speed of query processing. To this end, we present DPTC, a cost-effective and decompression-transparent approach designed to compress data pages, the basic storage unit of OLTP database systems. Specifically, (1) DPTC applies a row-wise decompression-oriented structure to track the first occurrence of redundant data in compressed data, which effectively supports the decompression of individual records from pages, thereby avoiding unwarranted decompression in record access. Moreover, (2) DPTC employs an in-page dynamic packing strategy, which determines the compression units based on the impact of each data reduction operation on the compression gains and eliminates gains-inefficient data reductions. Furthermore, (3) DPTC utilizes a SIMD-based mechanism that leverages the characteristics of operations within the decompression process to improve the decompression speed. Our evaluation results confirm that DPTC is efficient in terms of decompression speed and compression ratio. Within an OLTP database system, DPTC yields throughput improvements of up to 4.28 x in TPC-C and reduces latency by up to 33.3% for data point queries in a row-oriented storage engine.

As time series data become popular, their volume increases rapidly. Time series databases are designed for such data, and they process data in short slices, meaning that the compression units for compressors are small. How to compress the short slices of floating-points while reserving a high compression ratio and a high decompression speed remains a problem.To solve the problem, we propose a lossy compressor Machete. It uses an efficient hybrid encoder of Huffman encoding and variable length quantity (VLQ). Adaptive encoding selection makes it excel on short-slice data compression ratio, while the simple framework ensures fast decompression. We also find a limitation in VLQ and propose the optimal VLQ to further improve the compression ratio.Our evaluation on four real-world datasets shows that Machete outperforms state-of-the-art compressors by 32%−80% on compression ratio and achieves the fastest decompression speed on two datasets. When applied to a well-known time series database InfluxDB, Machete saves disk usage up to 79% and improves the query performance of the InfluxDB database by saving I/O.

Deduplication-based techniques are popular in backup storage systems for reducing data volume. To maximize data reduction, existing fine-grained deduplication approaches not only eliminate duplicate chunks but also delta-compress non-duplicate chunks as delta relative to their similar (base) chunks. However, each chunk may have multiple similar chunks, and delta compression usually only selects one of them as the base chunk, i.e., a one-to-one scheme. This scheme benefits the restore performance because it needs to read only one (instead of multiple) base chunk in decompressing delta chunks, while it also wastes the potential compressibility among other similar chunks.In this paper, we propose SuperDelta to further exploit compressibility across multiple similar chunks and to preserve the restore performance advantage of the one-to-one scheme as much as possible. It is based on three techniques. (1) To further eliminate redundancy among similar chunks, SuperDelta applies a "Multiple Referenced Base Chunks" (MRBC) scheme instead of the one-to-one scheme. It combines several similar pairs of chunks in delta encoding to recover possibly lost compressibility in "boundary shift" problems. (2) To avoid the negative side effects of MRBC on restore performance, SuperDelta introduces a rebase scheme to rebuild simple reference paths among duplicate and similar chunks. It significantly simplifies the restore workflow, but also costs slightly more storage space because of impacting the workflow of redundancy detection. (3) To compensate for the additional storage cost, SuperDelta applies a space-recycle scheme to remove derived data when they become old while ensuring the optimized restore performance of the latest backups.Experiments on four real-world backup datasets show that SuperDelta increases the overall compression ratio by 1.05~2.40 times than the traditional one-to-one fine-grained deduplication without significantly affecting the backup and restore throughput.

In recent years, the performance of hash indexes has been significantly improved by exploiting emerging persistent memory (PMem). However, the performance improvement of hash indexes mainly comes from exploiting the hardware features of PMem. Only a few studies optimize the hash index itself to fully exploit the potential of PMem. Interestingly, many of these studies improve the performance of write, but disregard the performance of read, of hash indexes on PMem. With extensive experimental evaluation, we find the major reason for inefficient read in the hash index on PMem is that the overhead of hash collision processing is expensive.To address that, we propose a novel Efficient Extendible Perfect Hashing (EEPH) on PMem-DRAM hybrid data layout to improve read performance of hash indexes. Specifically, we reduce the overhead of dynamic perfect hashing extension on PMem by combing extendible hashing. We then design a hybrid data layout to unlock the inherent read strengths of perfect hashing (i.e., zero collision). Last, we devise a complement move algorithm to efficiently guarantee the zero collision of perfect hashing when data move is conducted on PMem. We compare EEPH with the state-of-the-art hash indexes on PMem by conducting comprehensive experiments on several real-world read-intensive and read-skew workloads. The experimental results confirm the superiority of our EEPH as it achieves up to 2.21× higher throughput and about 1/3 of the 99th percentile latency than state-of-the-art hash indexes.

Images occupy a large amount of storage in data centers. To cope with the explosive growth of the image storage requirement, image compression techniques are devised to shrink the size of every single image at first. Furthermore, image deduplication methods are proposed to reduce the storage cost as they could be used to eliminate redundancy among images. However, state-of-the-art image deduplication methods either can only eliminate file-level coarse-grained redundancy or cannot guarantee lossless deduplication. In this work, we propose a new lossless image deduplication framework to eliminate fine-grained redundancy among images. It first decodes images to expose similarity, then eliminates fine-grained redundancy on the decoded data by delta compres-sion, and finally re-compresses the remaining data by image compression encoding. Based on this framework, we propose a novel lossless similarity-based deduplication (SBD) scheme for decoded image data (called imDedup). Specifically, imDedup uses a novel and fast sampling method (called Feature Map) to detect similar images in a two-dimensional way, which greatly reduces computation overhead. Meanwhile, it uses a novel delta encoder (called Idelta) which incorporates image compression encoding characteristics into deduplication to guarantee the remaining deduplicated image data to be friendly re-compressed via image encoding, which significantly improves the compression ratio. We implement a prototype of imDedup for JPEG images, and demonstrate its superiority on four datasets: Compared with exact image deduplication, imDedup achieves a 19%-38% higher compression ratio by efficiently eliminating fine-grained redundancy. Compared with the similarity detector and delta encoder of state-of-the-art SBD schemes running on the decoded image data, imDedup achieves a 1.8×-3.4× higher throughput and a 1.3 ×-1. 6 × higher compression ratio, respectively.

Fake news detection attracts many researchers’ attention due to the negative impacts on the society. Most existing fake news detection approaches mainly focus on semantic analysis of news’ contents. However, the detection performance will dramatically decrease when the content of news is short. In this paper, we propose a novelfake news detection multi-task learning (FDML)model based on the following observations: 1) some certain topics have higher percentages of fake news; and 2) some certain news authors have higher intentions to publish fake news. FDML model investigates the impact of topic labels for the fake news and introduce contextual information of news at the same time to boost the detection performance on the short fake news. Specifically, the FDML model consists of representation learning and multi-task learning parts to train the fake news detection task and the news topic classification task, simultaneously. As far as we know, this is the first fake news detection work that integrates the above two tasks. The experiment results show that the FDML model outperforms state-of-the-art methods on real-world fake news dataset.

Multiple data reduction techniques have been investigated to lower storage costs for a wide variety of customers. In this work, we focus on similarity-based delta compression, which calculates and stores the difference of very similar, but non-duplicate, chunks in storage systems. Delta compression is often implemented along with deduplication and has been shown to achieve a much higher compression ratio. Currently, the N-Transform method is the most popular and widely-used approach to generate features for data content (e.g. chunks) to detect similar candidates (and then apply delta compression). For delta compression systems, though, the throughput of N-Transform is often the bottleneck. Finesse is a high throughput variant of N-Transform, but it suffers from lower detection accuracy and compression ratio. The computation overhead of N-Transform consists of two parts: calculating the rolling hash across data and applying time-consuming transforms on each hash. In this work, we propose Odess, a fast resemblance detection approach, that uses a novel Content-Defined Sampling method to generate a much smaller proxy hash set and then applies transforms on this small hash set. This reduces the calculations in the transform step from being the bottleneck. Meanwhile, Odess also leverages the faster Gear hash to generate rolling hashes. Thus, Odess greatly reduces the computational overhead for resemblance detection while achieving high detection accuracy and high compression ratio. Our evaluation results show that Odess is ~ 5.4× (Finesse) and ~ 26.9× (N-Transform) faster (on average) at generating features for resemblance detection. When considering an end-to-end data reduction storage system, Odess increases throughput by ~ 1.36× (Finesse) and ~ 2.76× (N-Transform) while maintaining the compression ratio of N-Transform and increasing the compression ratio ~ 1.22× over Finesse.

Data reduction has become increasingly important in storage systems due to the explosive growth of digital data in the world that has ushered in the big data era. In this paper, we present DARE, a Deduplication-Aware Resemblance detection and Elimination scheme for compressing backup datasets that effectively combines data deduplication and delta compression to achieve high data reduction efficiency at low overhead. The main idea behind DARE is to employ a scheme, call Duplicate-Adjacency based Resemblance Detection (DupAdj), by considering any two data chunks to be similar (i.e., candidates for delta compression) if their respective adjacent data chunks are found to be duplicate in a deduplication system, and then further enhance the resemblance detection efficiency by an improved super-feature approach. Our experimental results based on real-world and synthetic backup datasets show that DARE achieves an additional data reduction by a factor of more than 2 (2X) on top of deduplication with very low overhead while nearly doubling the data restore performance of deduplication-only systems by supplementing delta compression to deduplication.