short-paper

Public Access

In Situ Summarization with VTK-m

Authors:

David Thompson,

Sebastien Jourdain,

Robert Maynard,

Ranga Raju Vatsavai,

Patrick O'LearyAuthors Info & Claims

ISAV'17: Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization

Pages 32 - 36

https://doi.org/10.1145/3144769.3144777

Published: 12 November 2017 Publication History

Abstract

Summarization and compression at current and future scales requires a framework for developing and benchmarking algorithms. We present a framework created by integrating existing, production-ready projects and provide timings of two particular algorithms that serve as exemplars for summarization: a wavelet-based data reduction filter and a generator for creating image-like databases of extracted features (isocontours in this case). Both support browser-based, post-hoc, interactive visualization of the summary for decision-making. A study of their weak-scaling on a distributed multi-GPU system is included.

References

[1]

T. Acharya and P.-S. Tsai. 2004. JPEG2000 Standard for Image Compression: Concepts, Algorithms and VLSI Architectures. Wiley-Interscience.

[2]

S. Ahern, A. Shoshani, K.-L. Ma, A. Choudhary, T. Critchlow, V. Pascucci, J. Ahrens, E. W. Bethel, H. Childs, J. Huang, K. Joy, Q. Koziol, G. Lofstead, J. Meredith, K. Moreland, G. Ostrouchov, M. Papka, V. Vishwanath, M. Wolf, N. Wright, and K. Wu. 2011. Scientific Discovery at the Exascale: Report from the DOE ASCR 2011 Workshop on Exascale Data Management, Analysis, and Visualization. Technical Report. Dept. of Energy, Office of Advanced Scientific Computing Research.

[3]

J. Ahrens, S. Jourdain, P. O'Leary, J. Patchett, D. H. Rogers, and M. Petersen. 2014. An image-based approach to extreme scale in situ visualization and analysis. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Press, 424--434.

Digital Library

[4]

J. Ao, S. Mitra, and B. Nutter. 2014. Fast and efficient loss-less image compression based on CUDA parallel wavelet tree encoding. In 2014 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI). IEEE, 21--24.

[5]

U. Ayachit. 2015. The ParaView Guide: A Parallel Visualization Application. Kitware, Inc., Clifton Park, NY.

[6]

U. Ayachit, A. Bauer, E. P. N. Duque, G. Eisenhauer, N. Ferrier, J. Gu, K. E. Jansen, B. Loring, Z. Lukic, S. Menon, D. Morozov, P. O'Leary, R. Ranjan, M. Rasquin, C. P. Stone, V. Vishwanath, G. H. Weber, B. Whitlock, M. Wolf, K. J. Wu, and E. W. Bethel. 2016. Performance Analysis, Design Considerations, and Applications of Extreme-scale In Situ Infrastructures. In Proc. of the International Conference for High Performance Computing, Networking, Storage and Analysis (SCâĂ&Zacute;16).

[7]

T. Ravindra Babu, M. Narasimha Murty, and S.V. Subrahmanya. 2013. Compression Schemes for Mining Large Datasets: A Machine Learning Perspective. Springer Verlag, London.

[8]

A. C. Bauer, H. Abbasi, J. Ahrens, H. Childs, B. Geveci, S. Klasky, K.Moreland, P. O'Leary, V. Vishwanath, and E. W. Bethel. 2016. In Situ Methods, Infrastructures, and Applications on High Performance Computing Platforms. Computer Graphics Forum 35, 3 (June 2016).

Digital Library

[9]

Wes Bethel, Brian Tierney, Jason lee, Dan Gunter, and Stephen Lau. 2000. Using High-speed WANs and Network Data Caches to Enable Remote and Distributed Visualization. In Supercomputing '00: Proceedings of the 2000 ACM/IEEE conference on Supercomputing. IEEE Computer Society, Dallas, Texas, United States. LBNL-45365.

[10]

H. Childs. 2007. Architectural challenges and solutions for petascale postprocessing. Journal of Physics: Conference Series 78, 012012 (2007).

[11]

Hank Childs, Eric Brugger, Brad Whitlock, Jeremy Meredith, Sean Ahern, David Pugmire, Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rübel, Marc Durant, Jean M. Favre, and Paul Navrátil. 2012. VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data. In High Performance Visualization--Enabling Extreme-Scale Scientific Insight. 357--372.

[12]

B. Chizi, L. Rokach, and O. Maimon. 2009. A Survey of Feature Selection Techniques (2 ed.).

[13]

I. Daubechies. 1992. Ten Lectures on Wavelets. SIAM, Philadelphia, PA.

[14]

Nathan Fabian, Kenneth Moreland, David Thompson, Andrew C. Bauer, Pat Marion, Berk Geveci, Michel Rasquin, and Kenneth E. Jansen. 2011. The ParaView Coprocessing Library: A Scalable, General Purpose In Situ Visualization Library. In IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV) 2011. Institute of Electrical and Electronics Engineers, 89--96.

[15]

A. Garcia and H.-W. Shen. 2005. GPU-based 3D wavelet reconstruction with tileboarding. The Visual Computer 21, 8 (2005), 755--763.

[16]

Al Globus. 1995. A Software Model for Visualization of Large Unsteady 3-D CFD Results. In 33rd Aerospace Sciences Meeting and Exhibit (AIAA). http://arc.aiaa.org/doi/abs/10.2514/6.1995-115

[17]

T. Goodale, G. Allen, G. Lanfermann, J. MassÃş, T. Radke, E. Seidel, and J. Shalf. 2003. The Cactus Framework and Toolkit: Design and Applications. In Vector and Parallel Processing - VECPAR '2002, 5th International Conference. Springer.

[18]

A. Goswami, Y. Tian, K. Schwan, F. Zheng, J. Young, M. Wolf, G. Eisenhauer, and S. Klasky. 2016. Landrush: Rethinking In-Situ Analysis for GPGPU Workflows. In Proceedings of the 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid). IEEE, Cartagena, Colombia.

Digital Library

[19]

Robert Haimes. 1995. Concurrent Distributed Visualization and Steering. In Parallel Computational Fluid Dynamics: Implementations and Results Using Parallel Computers.

[20]

Janet Jacobsen, E. Wes Bethel, Akhil Datta-Gupta, and Preston Holland. 1995. Petroleum Reservoir Simulation in a Virtual Environment. In Proceedings of the 13th Symposium on Reservoir Simulation (SPE). San Antonio TX, USA.

[21]

S. Jourdain, S. Wittenburg, T. Wright, A. Helser, and P. O'Leary. 2017. ArcticViewer. https://kitware.github.io/arctic-viewer/. (2017). [Online; accessed Aug 29, 2017].

[22]

A. Kageyama and T. Yamada. 2014. An approach to exascale visualization: Interactive viewing of in-situ visualization. Computer Physics Communications 185, 1 (2014), 79--85.

[23]

Matthew Larsen, Eric Brugger, Hank Childs, Jim Eliot, Kevin Griffin, and Cyrustit Harrison. 2015. Strawman: A Batch In Situ Visualization and Analysis Infrastructure for Multi-Physics Simulation Codes. In Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization.

Digital Library

[24]

S. Li, N. Marsaglia, V. Chen, C. Sewell, J. Clyne, and H. Childs. 2017. Achieving Portable Performance For Wavelet Compression Using Data Parallel Primitives. In Eurographics Symposium on Parallel Graphics and Visualization (EGPGV). Barcelona, Spain.

[25]

L. Lo, C. Sewell, and J. Ahrens. 2012. PISTON: A Portable Cross-Platform Framework for Data-Parallel Visualization Operators. In Proc. Eurographics Symp. Parallel Graphics and Visualization (EGPGV). 11--20.

[26]

Jay F Lofstead, Scott Klasky, Karsten Schwan, Norbert Podhorszki, and Chen Jin. 2008. Flexible I/O and integration for scientific codes through the adaptable I/O system (ADIOS). In Proceedings of the 6th international workshop on Challenges of large applications in distributed environments. ACM, 15--24.

Digital Library

[27]

K.-L. Ma. 1995. Runtime volume visualization of parallel CFD. In Proceedings of Parallel CFD Conference. 307--314.

[28]

J. S. Meredith, S. Ahern, D. Pugmire, and R. Sisneros. 2012. EAVL: The Extreme-Scale Analysis and Visualization Library. In Proc. Eurographics Symp. Parallel Graphics and Visualization. 21--30.

[29]

K. Moreland, B. King, R. Maynard, and Kwan-Liu Ma. 2012. Flexible Analysis Software for Emerging Architectures. In Proc. SC'12 Companion: High-Performance Computing, Networking Storage and Analysis. 821--826.

Digital Library

[30]

K. Moreland, C. Sewell, W. Usher, L.t. Lo, J. Meredith, D. Pugmire, J. Kress, H. Schroots, K.L. Ma, H. Childs, M. Larsen, C.M. Chen, R. Maynard, and B. Geveci. 2016. VTK-m: Accelerating the Visualization Toolkit for Massively Threaded Architectures. 36, 3 (May 2016), 48--58.

[31]

P. O'Leary, J. Ahrens, S. Jourdain, S. Wittenburg, D. H. Rogers, and M. Petersen. 2016. Cinema image-based in situ analysis and visualization of MPAS-ocean simulations. Parallel Comput. 55 (2016), 43--48.

Digital Library

[32]

Steven G. Parker, Christopher R. Johnson, and David Beazley. 1997. Computational Steering Software Systems and Strategies. IEEE Comput. Sci. Eng. 4, 4 (Oct. 1997), 50--59. https://doi.org/10.1109/99.641609

Digital Library

[33]

D. Salomon. 2007. Data Compression: The Complete Reference. Springer Verlag, London.

Digital Library

[34]

A. Tikhonova, C. D. Correa, and K.-L. Ma. 2010. Visualization by proxy: A novel framework for deferred interaction with volume data. 16, 6 (Nov. 2010), 1551--1559.

[35]

Tiankai Tu, Hongfeng Yu, Leonardo Ramirez-Guzman, Jacobo Bielak, Omar Ghattas, Kwan-Liu Ma, and David R. O'Hallaron. 2006. From Mesh Generation to Scientific Visualization: An End-to-End Approach to Parallel Supercomputing. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. DOI 10.1109/SC.2006.32.

[36]

Brad Whitlock, Jean M Favre, and Jeremy S Meredith. 2011. Parallel in situ coupling of simulation with a fully featured visualization system. In Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization. Eurographics Association, 101--109.

Digital Library

Cited By

Xing HAgrawal GRamnath RKloeckner AMoreira J(2022)GPU Adaptive In-situ Parallel Analytics (GAP)Proceedings of the International Conference on Parallel Architectures and Compilation Techniques10.1145/3559009.3569661(467-480)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3559009.3569661
Xing HAgrawal GRamnath RCuicchi CQualters IKramer W(2020)MoHAProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.5555/3433701.3433810(1-16)Online publication date: 9-Nov-2020
https://dl.acm.org/doi/10.5555/3433701.3433810
Xing HAgrawal GRamnath R(2020)MoHA: A Composable System for Efficient In-Situ Analytics on Heterogeneous HPC SystemsSC20: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41405.2020.00086(1-16)Online publication date: Nov-2020
https://doi.org/10.1109/SC41405.2020.00086
Show More Cited By

Index Terms

In Situ Summarization with VTK-m
1. Computing methodologies
  1. Distributed computing methodologies
    1. Distributed algorithms
  2. Parallel computing methodologies
    1. Parallel algorithms
2. Information systems
  1. Information systems applications
    1. Decision support systems
      1. Data analytics

Recommendations

Interactive visual summarization of multidimensional data
SMC'09: Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics

Visualization has become integral to the knowledge discovery process across various domains. However, challenges remain in the effective use of visualization techniques, especially when displaying, exploring and analyzing large, multidimensional ...
Hardware-Agnostic Interactive Exascale In Situ Visualization of Particle-In-Cell Simulations
PASC '23: Proceedings of the Platform for Advanced Scientific Computing Conference

The volume of data generated by exascale simulations requires scalable tools for analysis and visualization. Due to the relatively low I/O bandwidth of modern HPC systems, it is crucial to work as close as possible with simulated data via in situ ...
Leveraging NVIDIA Omniverse for In Situ Visualization
High Performance Computing
Abstract
Typical in situ visualization approaches involve rendering images of the simulation data in step with the simulation itself, using in situ visualization tools such as ParaView Catalyst, VisIt libsim, or SENSEI. For these approaches, one has to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISAV'17: Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization

November 2017

53 pages

ISBN:9781450351393

DOI:10.1145/3144769

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

U.S. Department of Energy
U.S. Army Small Business Technology Transfer Program Office and the Army Research Office

Conference

SC '17

Sponsor:

SIGHPC

SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 17, 2017

CO, Denver, USA

Acceptance Rates

ISAV'17 Paper Acceptance Rate 9 of 28 submissions, 32%;

Overall Acceptance Rate 23 of 63 submissions, 37%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
355
Total Downloads

Downloads (Last 12 months)43
Downloads (Last 6 weeks)8

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xing HAgrawal GRamnath RKloeckner AMoreira J(2022)GPU Adaptive In-situ Parallel Analytics (GAP)Proceedings of the International Conference on Parallel Architectures and Compilation Techniques10.1145/3559009.3569661(467-480)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3559009.3569661
Xing HAgrawal GRamnath RCuicchi CQualters IKramer W(2020)MoHAProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.5555/3433701.3433810(1-16)Online publication date: 9-Nov-2020
https://dl.acm.org/doi/10.5555/3433701.3433810
Xing HAgrawal GRamnath R(2020)MoHA: A Composable System for Efficient In-Situ Analytics on Heterogeneous HPC SystemsSC20: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41405.2020.00086(1-16)Online publication date: Nov-2020
https://doi.org/10.1109/SC41405.2020.00086
Warren RSoumagne JMu JTang HByna SDong BKoziol Q(2019)Analysis in the Data Path of an Object-Centric Data Management System2019 IEEE 26th International Conference on High Performance Computing, Data, and Analytics (HiPC)10.1109/HiPC.2019.00020(73-82)Online publication date: Dec-2019
https://doi.org/10.1109/HiPC.2019.00020

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents