research-article

Public Access

In-memory Integration of Existing Software Components for Parallel Adaptive Unstructured Mesh Workflows

Authors:

Cameron W. Smith,

Kenneth E. Jansen,

Mark S. ShephardAuthors Info & Claims

XSEDE16: Proceedings of the XSEDE16 Conference on Diversity, Big Data, and Science at Scale

Article No.: 34, Pages 1 - 6

https://doi.org/10.1145/2949550.2949650

Published: 17 July 2016 Publication History

Abstract

Reliable mesh-based simulations are needed to solve complex engineering problems. Mesh adaptivity can increase reliability by reducing discretization errors, but requires multiple software components to exchange information. Often, components exchange information by reading and writing a common file format. This file-based approach becomes a problem on massively parallel computers where filesystem bandwidth is a critical performance bottleneck. Our data stream and component interface approaches avoid the filesystem bottleneck. In this paper we present our approaches and their use within the PHASTA computational fluid dynamics solver and Albany multiphysics framework. Information exchange performance results are reported on up to 2048 cores of a BlueGene/Q system.

References

[1]

Albany website. http://gahansen.github.io/Albany, 2015.

[2]

Trilinos website. http://trilinos.org, 2015.

[3]

N. R. Adiga, G. Almasi, Y. Aridor, R. Barik, D. Beece, R. Bellofatto, G. Bhanot, R. Bickford, M. Blumrich, A. A. Bright, et al. An overview of the bluegene/l supercomputer. In Supercomputing, ACM/IEEE 2002 Conference, pages 60--60. IEEE, 2002.

Digital Library

[4]

S. Alam, R. Barrett, M. Bast, M. R. Fahey, J. Kuehn, C. McCurdy, J. Rogers, P. Roth, R. Sankaran, J. S. Vetter, et al. Early evaluation of ibm bluegene/p. In Proceedings of the 2008 ACM/IEEE conference on Supercomputing, page 23. IEEE Press, 2008.

Digital Library

[5]

F. Alauzet, X. Li, E. Seol, and M. Shephard. Parallel anisotropic 3d mesh adaptation by mesh modification. Engineering with Computers, 21(3):247--258, 2006.

Digital Library

[6]

W. K. Anderson, W. D. Gropp, D. K. Kaushik, D. E. Keyes, and B. F. Smith. Achieving high sustained performance in an unstructured mesh cfd application. In Proceedings of the 1999 ACM/IEEE conference on Supercomputing, page 69. ACM, 1999.

Digital Library

[7]

M. W. Beall, J. Walsh, and M. S. Shephard. A comparison of techniques for geometry access related to mesh generation. Engineering with Computers, 20:210--221, 2004.

Digital Library

[8]

J. C. Bennett, H. Abbasi, P.-T. Bremer, R. Grout, A. Gyulassy, T. Jin, S. Klasky, H. Kolla, M. Parashar, V. Pascucci, et al. Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for, pages 1--9. IEEE, 2012.

Digital Library

[9]

M. Besta and T. Hoefler. Fault tolerance for remote memory access programming models. In Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing, HPDC '14, pages 37--48, New York, NY, USA, 2014. ACM.

Digital Library

[10]

H. Bui, H. Finkel, V. Vishwanath, S. Habib, K. Heitmann, J. Leigh, M. Papka, and K. Harms. Scalable parallel i/o on a blue gene/q supercomputer using compression, topology-aware data aggregation, and subfiling. In Parallel, Distributed and Network-Based Processing (PDP), 2014 22nd Euromicro International Conference on, pages 107--111. IEEE, 2014.

Digital Library

[11]

C. Burstedde, L. C. Wilcox, and O. Ghattas. p4est: Scalable algorithms for parallel adaptive mesh refinement on forests of octrees. SIAM Journal on Scientific Computing, 33(3):1103--1133, 2011.

Digital Library

[12]

Co-Design. Crosscutting technologies for computing at the exascale, Draft Report, DOE, June, 2010. http://extremecomputing.labworks.org/crosscut/report.stm.

[13]

I. C. S. P. A. S. Committee, E. Open Group (Reading, I. of Electrical, and E. Engineers. Standard for Information Technology: Portable Operating System Interface (POSIX): Base Specifications. IEEE Std. 2013.

[14]

P. Frey and F. Alauzet. Anisotropic mesh adaptation for cfd computations. Computer Methods in Applied Mechanics and Engineering, 194(48-49):5068--5082, 2005. Unstructured Mesh Generation.

[15]

A. Y. Galimov, O. Sahni, R. T. Lahey, Jr., M. S. Shephard, D. A. Drew, and K. E. Jansen. Parallel adaptive simulation of a plunging liquid jet. Acta Mathematica Scientia, 30B:522--538, 2010.

[16]

R. A. Haring, M. Ohmacht, T. W. Fox, M. K. Gschwind, D. L. Satterfield, K. Sugavanam, P. W. Coteus, P. Heidelberger, M. A. Blumrich, R. W. Wisniewski, et al. The ibm blue gene/q compute chip. Micro, IEEE, 32(2):48--60, 2012.

Digital Library

[17]

M. A. Heroux, R. A. Bartlett, V. E. Howle, R. J. Hoekstra, J. J. Hu, T. G. Kolda, R. B. Lehoucq, K. R. Long, R. P. Pawlowski, E. T. Phipps, A. G. Salinger, H. K. Thornquist, R. S. Tuminaro, J. M. Willenbring, A. Williams, and K. S. Stanley. An overview of the trilinos project. ACM Trans. Math. Softw., 31(3):397--423, 2005.

Digital Library

[18]

D. A. Ibanez, E. S. Seol, C. W. Smith, and M. S. Shephard. Pumi: parallel unstructured mesh infrastructure. ACM Transactions on Mathematical Software, 2015.

Digital Library

[19]

Intel. Aurora fact sheet. http://www.intel.com/newsroom/assets/Intel_Aurora_factsheet.pdf, 2015. Accessed: 2016-04-27.

[20]

F. Isaila, J. Garcia, J. Carretero, R. Ross, and D. Kimpe. Making the case for reforming the i/o software stack of extreme-scale systems. Technical Report ANL/MCS-P5103-0314, Argonne National Laboratory, Chicago, IL, 2014.

[21]

T. Z. Islam, K. Mohror, S. Bagchi, A. Moody, B. R. de Supinski, and R. Eigenmann. Mcrengine: A scalable checkpointing system using data-aware aggregation and compression. In High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for, pages 1--11, Nov 2012.

Digital Library

[22]

K. Ko, A. Candel, L. Ge, A. Kabel, R. Lee, Z. Li, C. Ng, V. Rawat, G. Schussman, L. Xiao, et al. Advances in parallel electromagnetic codes for accelerator science and development. LINAC2010, Tsukuba, Japan, pages 1028--1032, 2010.

[23]

S. Lang, P. Carns, R. Latham, R. Ross, K. Harms, and W. Allcock. I/o performance challenges at leadership scale. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, page 40. ACM, 2009.

Digital Library

[24]

Y. J. Lo, S. Williams, B. Van Straalen, T. J. Ligocki, M. J. Cordery, N. J. Wright, M. W. Hall, and L. Oliker. Roofline model toolkit: A practical tool for architectural and program analysis. In High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation, pages 129--148. Springer, 2014.

[25]

M. Miller, J. Reus, R. Matzke, Q. Koziol, and A. Cheng. Smart libraries: Best sqe practices for libraries with emphasis on scientific computing. Proceedings of the Nuclear Explosives Code Developer's Conference, 2004.

[26]

R. O'Bara, M. Beall, and M. Shephard. Attribute management system for engineering analysis. Engineering with Computers, 18:339--351, 2002.

[27]

C. Ollivier-Gooch, L. Diachin, M. S. Shephard, T. Tautges, J. Kraftcheck, V. Leung, X.-J. Luo, and M. Miller. An interoperable, data-structure-neutral component for mesh query and manipulation. Transactions on Mathematical Software, 37(3):29:1--29:28, 2010.

Digital Library

[28]

M. Ortiz and J. Q. IV. Adaptive mesh refinement in strain localization problems. Computer Methods in Applied Mechanics and Engineering, 90(1-3):781 --804, 1991.

[29]

R. Radovitzky and M. Ortiz. Error estimation and adaptive meshing in strongly nonlinear dynamic problems. Computer Methods in Applied Mechanics and Engineering, 172(1-4):203--240, 1999.

[30]

M. Rasquin, C. Smith, K. Chitale, E. S. Seol, B. A. Matthews, J. L. Martin, O. Sahni, R. M. Loy, M. S. Shephard, and K. E. Jansen. Scalable Implicit Flow Solver for Realistic Wing Simulations with Flow Control. Computing in Science & Engineering, (6):13--21, 2014.

[31]

Roadmap. International exascale software project roadmap, DOE ASCR, january 2010. http://www.exascale.org/mediawiki/images/a/a1/Iesp-roadmap-draft-0.93-complete.pdf.

[32]

J. M. Rodriguez, O. Sahni, K. E. Jansen, and R. T. Lahey. A parallel adaptive mesh method for the numerical simulation of multiphase flows. In Proceedings of the 2008 Japan/US Seminar on Two-Phase Flow Dynamics, Santa Monica, CA, Sept. 2008.

[33]

O. Sahni, M. Zhou, M. S. Shephard, and K. E. Jansen. Scalable implicit finite element solver for massively parallel processing with demonstration to 160K cores. Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis - SC '09, page 1, 2009.

Digital Library

[34]

V. Sankaran, J. Sitaraman, A. Wissink, A. Datta, B. Jayaraman, M. Potsdam, D. Mavriplis, Z. Yang, D. O'Brien, H. Saberi, R. Cheng, N. Hariharan, and R. Strawn. Application of the helios computational platform to rotorcraft flowfields. In 48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, volume 1230, pages 1--28, 2010.

[35]

E. S. Seol and M. S. Shephard. Efficient distributed mesh data structure for parallel automated adaptive analysis. Engineering with Computers, 22(3-4):197--213, 2006.

[36]

C. W. Smith, M. Rasquin, D. Ibanez, K. E. Jansen, and M. S. Shephard. Application specific partition improvement. SIAM Journal on Scientific Computing, submitted, 2015.

[37]

C. H. Whiting and K. E. Jansen. A stabilized finite element method for the incompressible Navier-Stokes equations using a hierarchical basis. 35:93--116, 2001.

[38]

H. Yu, R. K. Sahoo, C. Howson, G. Almasi, J. Castaňos, M. Gupta, J. E. Moreira, J. Parker, T. Engelsiepen, R. B. Ross, et al. High performance file i/o for the blue gene/l supercomputer. In High-Performance Computer Architecture, 2006. The Twelfth International Symposium on, pages 187--196. IEEE, 2006.

[39]

F. Zhang, C. Docan, M. Parashar, S. Klasky, N. Podhorszki, and H. Abbasi. Enabling in-situ execution of coupled scientific workflow on multi-core platform. In Parallel & Distributed Processing Symposium (IPDPS), 2012 IEEE 26th International, pages 1352--1363. IEEE, 2012.

Digital Library

[40]

M. Zhou, O. Sahni, H. J. Kim, C. A. Figueroa, C. A. Taylor, M. S. Shephard, and K. E. Jansen. Cardiovascular flow simulation at extreme scale. Computational Mechanics, 46:71--82, 2010.

[41]

O. C. Zienkiewicz and J. Z. Zhu. The superconvergent patch recovery and a posteriori error estimates. part 1: The recovery technique. International Journal for Numerical Methods in Engineering, 33(7):1331--1364, 1992.

Cited By

Granzow BOberai AShephard M(2019)An automated approach for parallel adjoint-based error estimation and mesh adaptationEngineering with Computers10.1007/s00366-019-00890-2Online publication date: 4-Dec-2019
https://doi.org/10.1007/s00366-019-00890-2
Smith CGranzow BDiamond GIbanez DSahni OJansen KShephard M(2018)In‐memory integration of existing software components for parallel adaptive unstructured mesh workflowsConcurrency and Computation: Practice and Experience10.1002/cpe.451030:18Online publication date: 27-Apr-2018
https://doi.org/10.1002/cpe.4510

Recommendations

Parallel Processing of Adaptive Meshes with Load Balancing

Many scientific applications involve grids that lack a uniform underlying structure. These applications are often also dynamic in nature in that the grid structure significantly changes between successive phases of execution. In parallel computing ...
Nonsimplicial unstructured mesh generation
Solid model based parallel unstructured mesh generation

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

XSEDE16: Proceedings of the XSEDE16 Conference on Diversity, Big Data, and Science at Scale

July 2016

405 pages

ISBN:9781450347556

DOI:10.1145/2949550

General Chair:
Kelly Gaither
Texas Advanced Computing Center

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

SIGAPP: ACM Special Interest Group on Applied Computing
Xsede: Xsede
San Diego Supercomputer Center: San Diego Supercomputer Center
NICS: National Institute for Computational Sciences
University of Illinois: The University of Illinois at Urbana-Champaign

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

U.S. Department of Energy

Conference

XSEDE16

XSEDE16: Diversity, Big Data, and Science at Scale

July 17 - 21, 2016

Miami, USA

Acceptance Rates

Overall Acceptance Rate 129 of 190 submissions, 68%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
337
Total Downloads

Downloads (Last 12 months)109
Downloads (Last 6 weeks)11

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Granzow BOberai AShephard M(2019)An automated approach for parallel adjoint-based error estimation and mesh adaptationEngineering with Computers10.1007/s00366-019-00890-2Online publication date: 4-Dec-2019
https://doi.org/10.1007/s00366-019-00890-2
Smith CGranzow BDiamond GIbanez DSahni OJansen KShephard M(2018)In‐memory integration of existing software components for parallel adaptive unstructured mesh workflowsConcurrency and Computation: Practice and Experience10.1002/cpe.451030:18Online publication date: 27-Apr-2018
https://doi.org/10.1002/cpe.4510

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten