Recent years have witnessed a wide adoption of various lossy compression techniques to alleviate the burden on high-performance computing (HPC) systems that run large-scale scientific simulations producing large amounts of data. Peak signal-to-noise ratio (PSNR) is considered one of the most important indicators for measuring the distortion between reconstructed and original data and evaluating the performance of lossy compressors. However, the complex interplay between the error introduced during quantization and PSNR requires in-depth exploration to meet maximum compression potential and user-provided PSNR simultaneously. This paper aims to achieve this goal by exploring a novel quantization process to support a fixed-PSNR mode for a transform-based lossy compressor called DCTZ-F. We evaluate DCTZ-F with six real-world scientific datasets from several solvers in FLASH, a multi-physics application code. Our experimental results show that DCTZ-F can generate up to three times the compression ratio than SZ with fixed-PSNR mode while meeting the user-defined PSNR.

High-performance computing (HPC) systems that run scientific simulations of significance produce a large amount of data during runtime. Transferring or storing such big datasets causes a severe I/O bottleneck and a considerable storage burden. Applying compression techniques, particularly lossy compressors, can reduce the size of the data and mitigate such overheads. Unlike lossless compression algorithms, error-controlled lossy compressors could significantly reduce the data size while respecting the user-defined error bound. DCTZ is one of the transform-based lossy compressors with a highly efficient encoding and purpose-built error control mechanism that accomplishes high compression ratios with high data fidelity. However, since DCTZ quantizes the DCT coefficients in the frequency domain, it may only partially control the relative error bound defined by the user. In this paper, we aim to improve the compression quality of DCTZ. Specifically, we propose a preconditioning method based on level offsetting and scaling to control the magnitude of input of the DCTZ framework, thereby enforcing stricter error bounds. We evaluate the performance of our method in terms of compression ratio and rate distortion with real-world HPC datasets. Our experimental result shows that our method can achieve a higher compression ratio than other state-of-the-art lossy compressors with a tighter error bound while precisely guaranteeing the user-defined error bound.