research-article

Assessing the effects of data compression in simulations using physically motivated metrics

Authors:

Al WegenerAuthors Info & Claims

SC '13: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

Article No.: 76, Pages 1 - 12

https://doi.org/10.1145/2503210.2503283

Published: 17 November 2013 Publication History

Get Access

Abstract

This paper examines whether lossy compression can be used effectively in physics simulations as a possible strategy to combat the expected data-movement bottleneck in future high performance computing architectures. We show that, for the codes and simulations we tested, compression levels of 3--5X can be applied without causing significant changes to important physical quantities.

Rather than applying signal processing error metrics, we utilize physics-based metrics appropriate for each code to assess the impact of compression. We evaluate three different simulation codes: a Lagrangian shock-hydrodynamics code, an Eulerian higher-order hydrodynamics turbulence modeling code, and an Eulerian coupled laser-plasma interaction code. We compress relevant quantities after each time-step to approximate the effects of tightly coupled compression and study the compression rates to estimate memory and disk-bandwidth reduction. We find that the error characteristics of compression algorithms must be carefully considered in the context of the underlying physics being modeled.

References

[1]

R. L. Berger, B. F. Lasinski, A. B. Langdon, T. B. Kaiser, B. B. Afeyan, B. I. Cohen, C. H. Still, and E. A. Williams. Influence of spatial and temporal laser beam smoothing on stimulated brillouin scattering in filamentary laser light. Physical Review Letters, 75(6):1078--1081, Aug. 1995.

Crossref

Google Scholar

[2]

M. Burtscher and P. Ratanaworabhan. High throughput compression of double-precision floating-point data. In Data Compression Conference, pages 293--302, Mar. 2007.

Digital Library

Google Scholar

[3]

W. H. Cabot and A. W. Cook. Reynolds number effects on Rayleigh-Taylor instability with possible implications for type-Ia supernovae. Nature Physics, 2:562--568, 2006.

Crossref

Google Scholar

[4]

E. J. Candès and M. B. Wakin. An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2):21--30, Mar. 2008.

Crossref

Google Scholar

[5]

A. W. Cook, W. Cabot, and P. L. Miller. The mixing transition in Rayleigh--Taylor instability. Journal of Fluid Mechanics, 511:333--362, 2004.

Crossref

Google Scholar

[6]

A. W. Cook, W. H. Cabot, M. L. Welcome, P. L. Williams, B. J. Miller, B. R. de Supinski, and R. K. Yates. Tera-scalable algorithms for variable-density elliptic hydrodynamics with spectral accuracy. In ACM/IEEE Conference on Supercomputing, page 60, 2005.

Digital Library

Google Scholar

[7]

D. L. Donoho. Compressed sensing. IEEE Transactions on Information Theory, 52(4):1289--1306, Apr. 2006.

Digital Library

Google Scholar

[8]

N. Fout and K.-L. Ma. An adaptive prediction-based approach to lossless compression of floating-point volume data. IEEE Transactions on Visualization and Computer Graphics, 18(12):2295--2304, Dec. 2012.

Digital Library

Google Scholar

[9]

N. Huebbe and J. Kunkel. Reducing the HPC-datastorage footprint with MAFISC multidimensional adaptive filtering improved scientific data compression. Computer Science Research and Development Journal, 28(2--3):231--239, May 2012.

Digital Library

Google Scholar

[10]

N. Huebbe, A. Wegener, J. Kunkel, Y. Ling, and T. Ludwig. Evaluating lossy compression on climate data. In International Supercomputing Conference, pages 343--356, June 2013.

Crossref

Google Scholar

[11]

L. Ibarria, P. Lindstrom, J. Rossignac, and A. Szymczak. Out-of-core compression and decompression of large n-dimensional scalar fields. Computer Graphics Forum, 22(3):343--348, 2003.

Crossref

Google Scholar

[12]

I. Karlin, A. Bhatele, B. L. Chamberlain, J. Cohen, Z. Devito, M. Gokhale, R. Haque, R. Hornung, J. Keasler, D. Laney, E. Luke, S. Lloyd, J. McGraw, R. Neely, D. Richards, M. Schulz, C. H. Still, F. Wang, and D. Wong. LULESH programming model and performance ports overview. Technical Report LLNL-TR-608824, Lawrence Livermore National Laboratory, Dec. 2012.

Crossref

Google Scholar

[13]

I. Karlin, A. Bhatele, J. Keasler, B. L. Chamberlain, J. Cohen, Z. DeVito, R. Haque, D. Laney, E. Luke, F. Wang, D. Richards, M. Schulz, and C. Still. Exploring traditional and emerging parallel programming models using a proxy application. In IEEE International Parallel & Distributed Processing Symposium, pages 919--932, May 2013.

Digital Library

Google Scholar

[14]

J. Keasler and R. Hornung. Hydrodynamics challenge problem. Technical Report LLNL-TR-490254, Lawrence Livermore National Laboratory, 2010.

Google Scholar

[15]

P. Kogge, K. Bergman, S. Borkar, D. Campbell, et al. Exascale computing study: Technology challenges in achieving exascale systems, Sept. 2008.

Google Scholar

[16]

S. Lakshminarasimhan, N. Shah, S. Ethier, S. Klasky, R. Latham, R. Ross, and N. F. Samatova. Compressing the incompressible with ISABELA: In-situ reduction of spatio-temporal data. In Euro-Par Parallel Processing, Lecture Notes in Computer Science, pages 366--379. Springer, 2011.

Digital Library

Google Scholar

[17]

S. Langer, B. Still, T. Bremer, D. Hinkel, B. Langdon, J. A. Levine, and E. A. Williams. Cielo full-system simulations of multi-beam laser-plasma interaction in NIF experiments. In Proceedings of the 53rd Cray User Group Meeting, 2011.

Google Scholar

[18]

H. Lehmann and B. Jung. In-situ data compression for flow simulation in porous media. In Parallel & Distributed Processing Techniques & Applications, 2012.

Google Scholar

[19]

P. Lindstrom. fpzip version 1.0.1, 2008. https://computation.llnl.gov/casc/fpzip/.

Google Scholar

[20]

P. Lindstrom and M. Isenburg. Fast and efficient compression of floating-point data. IEEE Transactions on Visualization and Computer Graphics, 12(5):1245--1250, 2006.

Digital Library

Google Scholar

[21]

S. K. Moore. Multicore is bad news for supercomputers. IEEE Spectrum, 45(11):15, 2008.

Digital Library

Google Scholar

[22]

E. I. Moses. Overview of the National Ignition Facility. Fusion Science and Technology, 54(2):361--366, 2008.

Crossref

Google Scholar

[23]

E. I. Moses, R. N. Boyd, B. A. Remington, C. J. Keane, and R. Al-Ayat. The National Ignition Facility: Ushering in a new age for high energy density science. Physics of Plasmas, 16(041006):1--13, 2009.

Google Scholar

[24]

S. Muraki. Approximation and rendering of volume data using wavelet transforms. In IEEE Visualization, pages 21--28, 1992.

Digital Library

Google Scholar

[25]

S. Muraki. Multiscale volume representation by a dog wavelet. IEEE Transactions on Visualization and Computer Graphics, 1(2):109--116, 1995.

Digital Library

Google Scholar

[26]

R. Murphy. On the effects of memory latency and bandwidth on supercomputer application performance. IEEE International Symposium on Workload Characterization, pages 34--43, 2007.

Digital Library

Google Scholar

[27]

E. Schendel, Y. Jin, N. Shah, J. Chen, C. S. Chang, S.-H. Ku, S. Ethier, S. Klasky, R. Latham, R. Ross, and N. Samatova. ISOBAR preconditioner for effective and high-throughput lossless data compression. In IEEE International Conference on Data Engineering, pages 138--149, 2012.

Digital Library

Google Scholar

[28]

C. H. Still, R. L. Berger, A. B. Langdon, D. E. Hinkel, L. J. Suter, and E. A. Williams. Filamentation and forward Brillouin scatter of entire smoothed and aberrated laser beams. Physics of Plasmas, 7(5):2023--2032, 2000.

Crossref

Google Scholar

[29]

E. Tasker, R. Brunino, N. Mitchell, D. Michielsen, S. Hopton, F. Pearce, G. Bryan, and T. Theuns. A test suite for quantitative comparison of hydrodynamics codes in astrophysics. Monthly Notices of the Royal Astronomical Society, 390(3):1267--1281, 2008.

Crossref

Google Scholar

[30]

A. Wegener. Adaptive compression and decompression of bandlimited signals, Mar. 2006. http://www.patentlens.net/patentlens/patent/US_7009533/.

Google Scholar

[31]

A. Wegener. Block floating point compression of signal data, Oct. 2012. http://www.patentlens.net/patentlens/patent/US_8301803/.

Google Scholar

[32]

A. Wegener. Universal numerical encoder and profiler reduces computing memory wall with software, FPGA, and SoC implementations. In IEEE Data Compression Conference, page 528, Mar. 2013.

Digital Library

Google Scholar

[33]

A. Wegener, N. Chandra, Y. Ling, R. Senzig, and R. Herfkens. Effects of fixed-rate CT projection data compression on perceived and measured CT image quality. In SPIE Medical Imaging, volume 7627, Feb. 2010.

Crossref

Google Scholar

[34]

M. L. Wilkins. Methods in Computational Physics. Academic Press, 1964.

Google Scholar

[35]

J. Woodring, S. Mniszewski, C. Brislawn, D. DeMarle, and J. Ahrens. Revisiting wavelet compression for large-scale climate data using JPEG 2000 and ensuring data precision. In IEEE Large Data Analysis and Visualization, pages 31--38, 2011.

Crossref

Google Scholar

Cited By

View all

Lindstrom PHittinger JDiffenderfer JFox AOsei-Kuffuor DBanks J(2024)ZFP: A compressed array representation for numerical computationsThe International Journal of High Performance Computing Applications10.1177/10943420241284023Online publication date: 23-Oct-2024
https://doi.org/10.1177/10943420241284023
Fridman YTamir GOren G(2023)Portability and Scalability of OpenMP Offloading on State-of-the-Art AcceleratorsHigh Performance Computing10.1007/978-3-031-40843-4_28(378-390)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-40843-4_28
Bhatia HHoang DMorrical NPascucci VBremer PLindstrom P(2022)AMM: Adaptive Multilinear MeshesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.3165392(1-1)Online publication date: 2022
https://doi.org/10.1109/TVCG.2022.3165392
Show More Cited By

Index Terms

Assessing the effects of data compression in simulations using physically motivated metrics
1. Information systems
  1. Data management systems
    1. Data structures
      1. Data layout
        Data compression
  2. Information retrieval
    1. Evaluation of retrieval results
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Error-correcting codes

Recommendations

Assessing the effects of data compression in simulations using physically motivated metrics
SC13 --The International Conference for High Performance Computing, Networking, Storage and Analysis

This paper examines whether lossy compression can be used effectively in physics simulations as a possible strategy to combat the expected data-movement bottleneck in future high performance computing architectures. We show that, for the codes and ...
Vertex Data Compression through Vector Quantization

Rendering geometrically detailed 3D models requires the transfer and processing of large amounts of triangle and vertex geometry data. Compressing the geometry bitstream can reduce bandwidth requirements and alleviate transmission bottlenecks. In this ...
Second compression for pixelated images under edge-based compression algorithms: JPEG-LS as an example

This paper details the examination of a particular case of data compression, where the compression algorithm removes the redundancy from data, which occurs when edge-based compression algorithms compress (previously compressed) pixelated images. The newly ...

Reviews

Reviewer: Pierre Jouvelot

There is an imbalance between the fast speed of central processing units (CPUs) and the long access time of memory subsystems. This is the so-called "memory wall," and it significantly limits potential performance in current and future computer architectures. One could tackle this issue by using core or disk memory compression to reduce the volume of exchanged data. Using lossless compression would preserve computational accuracy, but the expected payoff would be much lower than with more efficient lossy compression schemes. The authors of this paper discuss the practical impact of such losses on actual computations and propose APAX and fpzip, two predictive coders specialized for floating-point data. These two approaches are evaluated to determine how they affect the end results of three simulation benchmarks-LULESH, Miranda, and pF3D-which represent different domains of physics, such as hydrodynamics and laser-plasma interactions. The authors emphasize that physically meaningful differences between compressed and uncompressed runs should be evaluated, in addition to traditional measures such as mean square errors. Comparisons of data are based on, for instance, the symmetry of the computed fields, the structure of intensity histograms, the height of turbulent mixing layers, and the spectrum of perturbations as a function of spatial frequency. Detailed analyses show that compression ratios of up to four times can be used most of the time without jeopardizing the practical validity of these simulations. This easy-to-read paper should be of value to scientific computing specialists and computer architects interested in achieving maximal performance on high-performance computing systems. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SC '13: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

November 2013

1123 pages

ISBN:9781450323789

DOI:10.1145/2503210

General Chair:
William Gropp
University of Illinois at Urbana-Champaign, Urbana, Illinois
,
Program Chair:
Satoshi Matsuoka
Tokyo Institute of Technology, Tokyo, Japan

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SC13

Sponsor:

SIGHPC
SIGARCH
IEEE-CS

SC13: International Conference for High Performance Computing, Networking, Storage and Analysis

November 17 - 21, 2013

Colorado, Denver

Acceptance Rates

SC '13 Paper Acceptance Rate 91 of 449 submissions, 20%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
300
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)6

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lindstrom PHittinger JDiffenderfer JFox AOsei-Kuffuor DBanks J(2024)ZFP: A compressed array representation for numerical computationsThe International Journal of High Performance Computing Applications10.1177/10943420241284023Online publication date: 23-Oct-2024
https://doi.org/10.1177/10943420241284023
Fridman YTamir GOren G(2023)Portability and Scalability of OpenMP Offloading on State-of-the-Art AcceleratorsHigh Performance Computing10.1007/978-3-031-40843-4_28(378-390)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-40843-4_28
Bhatia HHoang DMorrical NPascucci VBremer PLindstrom P(2022)AMM: Adaptive Multilinear MeshesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.3165392(1-1)Online publication date: 2022
https://doi.org/10.1109/TVCG.2022.3165392
Chen QCao JXia Y(2022)Physics-Enhanced PCA for Data Compression in Edge DevicesIEEE Transactions on Green Communications and Networking10.1109/TGCN.2022.31716816:3(1624-1634)Online publication date: Sep-2022
https://doi.org/10.1109/TGCN.2022.3171681
Underwood RBessac JDi SCappello F(2022)Understanding the Effects of Modern Compressors on the Community Earth Science Model2022 IEEE/ACM 8th International Workshop on Data Analysis and Reduction for Big Scientific Data (DRBSD)10.1109/DRBSD56682.2022.00006(1-10)Online publication date: Nov-2022
https://doi.org/10.1109/DRBSD56682.2022.00006
Kolomenskiy DOnishi RUehara H(2022)WaveRange: wavelet-based data compression for three-dimensional numerical simulations on regular gridsJournal of Visualization10.1007/s12650-021-00813-825:3(543-573)Online publication date: 1-Jan-2022
https://doi.org/10.1007/s12650-021-00813-8
Devarajan HKougkas ALogan LSun X(2020)HCompress: Hierarchical Data Compression for Multi-Tiered Storage Environments2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS47924.2020.00064(557-566)Online publication date: May-2020
https://doi.org/10.1109/IPDPS47924.2020.00064
Pinard AHammerling DBaker A(2020)Assessing Differences in Large Spatio-temporal Climate Datasets with a New Python package2020 IEEE International Conference on Big Data (Big Data)10.1109/BigData50022.2020.9378100(2699-2707)Online publication date: 10-Dec-2020
https://doi.org/10.1109/BigData50022.2020.9378100
Baker AHammerling DTurton T(2019)Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation dataComputer Graphics Forum10.1111/cgf.1370738:3(517-528)Online publication date: 10-Jul-2019
https://doi.org/10.1111/cgf.13707
Hoang DKlacansky PBhatia HBremer PLindstrom PPascucci V(2019)A Study of the Trade-off Between Reducing Precision and Reducing Resolution for Data Analysis and VisualizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.286485325:1(1193-1203)Online publication date: Jan-2019
https://doi.org/10.1109/TVCG.2018.2864853
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations