Technical Papers Archive

Dynamic Quality Metric Oriented Error Bounded Lossy Compression for Scientific Datasets

Authors: Jinyang Liu (University of California, Riverside; Argonne National Laboratory (ANL)); Sheng Di (Argonne National Laboratory (ANL)); Kai Zhao (University of Alabama, Birmingham); Xin Liang (University of Kentucky); Zizhong Chen (University of California, Riverside); and Franck Cappello (Argonne National Laboratory (ANL))

Abstract: Error-bounded lossy compression has been considered a promising solution to address the big-data issue for scientific application. However, the existing lossy compressors are all developed on fixed designs which cannot adapt to diverse quality metrics favored by different users. In this paper, we propose QoZ, a dynamic quality metric oriented error bounded lossy compressor. Our key contributions include: (1) We propose a highly-parameterized multi-level interpolation based data predictor which significantly improves the compression quality with the same compressed size. (2) We design the lossy compression framework QoZ with the predictor proposed, which can auto-tune parameters and optimize the compression based on user-specified quality metrics. (3) We evaluate QoZ carefully by comparing it with multiple state-of-the-arts on real-world scientific application datasets. Experiments show that, compared with the second best, QoZ achieves up to 70% compression ratio improvement under the same error bound or 150%(270%) compression ratio improvement under the same PSNR(SSIM).




Back to Technical Papers Archive Listing