research-article

A space-time parallel algorithm with adaptive mesh refinement for computational fluid dynamics

Authors:

Joshua Christopher,

Robert D. Falgout,

Jacob B. Schroder,

Stephen M. Guzik,

Xinfeng GaoAuthors Info & Claims

Computing and Visualization in Science, Volume 23, Issue 1-4

https://doi.org/10.1007/s00791-020-00334-1

Published: 01 December 2020 Publication History

Abstract

This paper describes a space-time parallel algorithm with space-time adaptive mesh refinement (AMR). AMR with subcycling is added to multigrid reduction-in-time (MGRIT) in order to provide solution efficient adaptive grids with a reduction in work performed on coarser grids. This algorithm is achieved by integrating two software libraries: XBraid (Parallel time integration with multigrid. https://computation.llnl.gov/projects/parallel-timeintegration-multigrid) and Chombo (Chombo software package for AMR applications—design document, 2014). The former is a parallel time integration library using multigrid and the latter is a massively parallel structured AMR library. Employing this adaptive space-time parallel algorithm is Chord (Comput Fluids 123:202–217, 2015), a computational fluid dynamics (CFD) application code for solving compressible fluid dynamics problems. For the same solution accuracy, speedups are demonstrated from the use of space-time parallelization over the time-sequential integration on Couette flow and Stokes’ second problem. On a transient Couette flow case, at least a

1.5 \times

speedup is achieved, and with a time periodic problem, a speedup of up to

13.7 \times

over the time-sequential case is obtained. In both cases, the speedup is achieved by adding processors and exploring additional parallelization in time. The numerical experiments show the algorithm is promising for CFD applications that can take advantage of the time parallelism. Future work will focus on improving the parallel performance and providing more tests with complex fluid dynamics to demonstrate the full potential of the algorithm.

References

[1]

AMReX: Software framework for block structured AMR. https://amrex-codes.github.io/amrex/. Accessed 3 Oct 2018

[2]

CASTRO: An adaptive mesh, astrophysical radiation hydrodynamics simulation code. https://amrex-astro.github.io/Castro/. Accessed 16 May 2019

[3]

Overture: An object-oriented toolkit for solving partial differential equations in complex geometry. https://www.overtureframework.org/. Accessed 16 May 2019

[4]

p4est: Parallel adaptive mesh refinement on forests of octrees. http://www.p4est.org/. Accessed 16 May 2019

[5]

SAMRAI: Structured adaptive mesh refinement application infrastructure. https://computation.llnl.gov/projects/samrai. Accessed 3 Oct 2018

[6]

XBraid: Parallel time integration with multigrid. https://computation.llnl.gov/projects/parallel-time-integration-multigrid. Accessed 24 Sep 2018

[7]

Adams, M., Colella, P., Graves, D.T., Johnson, J.N., Johansen, H.S., Keen, N.D., Ligocki, T.J., Martin, D.F., McCorquodale, P.W., Modiano, D., Schwartz, P.O., Sternberg, T.D., Van Straalen, B.: Chombo software package for AMR applications - design document. Technical Report LBNL-6616E, Lawrence Berkeley National Laboratory (2014)

[8]

Anderson J Modern Compressible Flow: With Historical Perspective 2002 3 Boston McGraw-Hill Education

[9]

Batchelor GK An Introduction to Fluid Dynamics 2000 Cambridge Cambridge University Press

[10]

Brandt, A.: Multi-level adaptive computations in fluid dynamics. In: Technical Report AIAA-79-1455. AIAA, Williamsburg, VA (1979)

[11]

Briggs WL, Henson VE, and McCormick SF A Multigrid Tutorial 2000 2 Philadelphia Society for Industrial and Applied Mathematics

[12]

Christopher, J., Gao, X., Guzik, S.M., Falgout, R., Schroder, J.: Parallel in time for a fully space-time adaptive mesh refinement algorithm. In: AIAA Scitech 2020 Forum. American Institute of Aeronautics and Astronautics, Orlando, FL (2020).

[13]

Christopher, J., Gao, X., Guzik, S.M., Falgout, R.D., Schroder, J.B.: Space-time adaptivity with multigrid reduction in time for the compressible navier-stokes equations. In: 19th Copper Mountain Conference On Multigrid Methods, pp. 10 (2019)

[14]

Colella P, Dorr MR, Hittinger JAF, and Martin DF High-order finite-volume methods in mapped coordinates JCP 2011 230 2952-2976

[15]

Colella P and Sekora MD A limiter for PPM that preserves accuracy at smooth extrema J. Comput. Phys. 2008 227 15 7069-7076

[16]

Day MS and Bell JB Numerical simulation of laminar reacting flows with complex chemistry Combust. Theory Model. 2000 4 535-556

[17]

De Sterck H, Falgout RD, Howse AJM, MacLachlan SP, and Schroder JB Parallel-in-time multigrid with adaptive spatial coarsening for the linear advection and inviscid burgers equations SIAM J. Sci. Comput. 2017 41 A538

[18]

Dubey A, Almgren A, Bell J, Berzins M, Brandt S, Bryan G, Colella P, Graves D, Lijewski M, Löffler F, O’Shea B, Schnetter E, Van Straalen B, and Weide K A survey of high level frameworks in block-structured adaptive mesh refinement packages J. Parallel Distrib. Comput. 2014 74 12 3217-3227

[19]

Falgout RD, Friedhoff S, Kolev TV, MacLachlan SP, and Schroder JB Parallel time integration with multigrid SIAM J. Sci. Comput 2014 36 C635-C661

[20]

Falgout RD, Friedhoff S, Kolev TV, MacLachlan SP, and Schroder JB Parallel time integration with multigrid SIAM J. Sci. Comput. 2014 36 6 C635-C661

[21]

Falgout RD, Friedhoff S, Vandewalle S, Kolev TV, MacLachlan SP, and Schroder JB Multigrid methods with space-time concurrency Comput. Vis. Sci. 2017 18 4–5 123-143

[22]

Falgout, R.D., Katz, A., Kolev, T.V., Schroder, J., Wissink, A.M., Yang, U.: Parallel time integration with multigrid reduction for a compressible fluid dynamics application. Technical report Lawrence Livermore National Lab (2014)

[23]

Falgout, R.D., Katz, A., Kolev, T.V., Schroder, J.B., Wissink, A., Yang, U.M.: Parallel time integration with multigrid reduction for a compressible fluid dynamics application. Technical Report LLNL-JRNL-663416, Lawrence Livermore National Laboratory (2015)

[24]

Falgout, R.D., Manteuffel, T.A., Schroder, J.B., Southworth, B.: Parallel-in-time for moving meshes. Technical Report LLNL-TR-681918, Lawrence Livermore National Laboratory (2016)

[25]

Friedhoff, S., Hahne, J., Kulchytska-Ruchka, I., Schöps, S.: Exploring parallel-in-time approaches for eddy current problems. arXiv preprint arXiv:1810.13263 (2018)

[26]

Fryxell B, Olson K, Ricker P, Timmes FX, Zingale M, Lamb DQ, MacNeice P, Rosner R, Truran JW, and Tufo H FLASH: an adaptive mesh hydrodynamics code for modeling astrophysical thermonuclear flashes ApJS 2000 131 1 273

[27]

Gander MJ Carraro T, Geiger M, Körkel S, and Rannacher R 50 years of time parallel time integration Multiple Shooting and Time Domain Decomposition 2015 Berlin Springer 69-114

[28]

Gander MJ and Neumüller M Analysis of a new space-time parallel multigrid algorithm for parabolic problems SIAM J. Sci. Comput. 2016 38 4 A2173-A2208

[29]

Gao, X.: A parallel solution-adaptive method for turbulent non-premixed combusting flows. Ph.D. thesis, University of Toronto (2008)

[30]

Gao, X., Guzik, S.M., Olschanowsky, C.: A high-performance fourth-order finite-volume algorithm for compressible viscous flows on structured grids. Submitted to AIAA J

[31]

Gao, X., Guzik, S.M.J.: A fourth-order scheme for the compressible navier-stokes equations. In: AIAA 2015-0298, 53rd AIAA Aerospace Sciences Meeting (2015)

[32]

Gao, X., Guzik, S.M.J., Colella, P.: Fourth order boundary treatment for viscous fluxes on cartesian grid finite-volume methods. In: AIAA 2014-1277, 52nd AIAA Aerospace Sciences Meeting (2014)

[33]

Gao X, Northrup S, and Groth C Parallel solution-adaptive method for two-dimensional non-premixed combusting flows Progress Comput. Fluid. Dyn. Int. J. 2011 11 2 76-95

[34]

Gao, X., Owen, L.D., Guzik, S.M.: A parallel adaptive numerical method with generalized curvilinear coordinate transformation for compressible navier-stokes equations. Int. J. Numer. Meth. Fluids, Accepted

[35]

Gao, X., Owen, L.D., Guzik, S.M.: A high-order finite-volume method for combustion. In: AIAA 2016-1808, 54th AIAA Aerospace Sciences Meeting (2016)

[36]

Garnier, E., Adams, N., Sagaut, P.: Large Eddy Simulation for Compressible Flows. Scientific Computation. Springer Netherlands (2009).

[37]

Georgiadis, N.J., Rizzetta, D.P., Fureby, C.: Large-eddy simulation: Current capabilities, recommended practices, and future research pp. 27 (2009)

[38]

Groth, C., De Zeeuw, D., Powell, K., Gombosi, T., Stout, Q.: A parallel solution-adaptive scheme for ideal magnetohydrodynamics. In: 14th Computational Fluid Dynamics Conference. American Institute of Aeronautics and Astronautics, Norfolk, VA, U.S.A. (1999).

[39]

Günther S, Gauger NR, and Schroder JB A non-intrusive parallel-in-time adjoint solver with the XBraid library Comput. Vis. Sci. 2018 19 3–4 85-95

[40]

Günther S, Gauger NR, and Schroder JB A non-intrusive parallel-in-time approach for simultaneous optimization with unsteady PDEs Optim. Methods Softw. 2018 34 1306

[41]

Günther, S., Ruthotto, L., Schroder, J., Cyr, E., Gauger, N.: Layer-parallel training of deep residual neural networks. SIAM Journal on Data Science (2019 (submitted)). ArXiv preprint arXiv:1812.04352

[42]

Guzik SM, Gao X, and Olschanowsky C A high-performance finite-volume algorithm for solving partial differential equations governing compressible viscous flows on structured grids Comput. Math Appl. 2016 72 2098-2118

[43]

Guzik SM, Gao X, Owen LD, McCorquodale P, and Colella P A freestream-preserving fourth-order finite-volume method in mapped coordinates with adaptive-mesh refinement Comput. Fluids 2015 123 202-217

[44]

Guzik, S.M.J., McCorquodale, P., Colella, P.: A freestream-preserving high-order finite-volume method for mapped grids with adaptive-mesh refinement. In: AIAA 2012-0574, 50th AIAA Aerospace Sciences Meeting (2012)

[45]

Hessenthaler A, Nordsletten D, Röhrle O, Schroder J, and Falgout R Convergence of the multigrid-reduction-in-time algorithm for the linear elasticity equations Numer. Linear Algebra Appl. 2018 25 3 e2155

[46]

Horton G and Vandewalle S A space-time multigrid method for parabolic partial differential equations SIAM J. Sci. Comput. 1995 16 4 848-864

[47]

Howse AJ, Sterck HD, Falgout RD, MacLachlan S, and Schroder J Parallel-in-time multigrid with adaptive spatial coarsening for the linear advection and inviscid burgers equations SIAM J. Sci. Comput. 2019 41 1 A538-A565

[48]

Langer U, Matculevich S, and Repin S Guaranteed error bounds and local indicators for adaptive solvers using stabilised space-time IgA approximations to parabolic problems Comput. Math. Appl. 2019 78 8 2641-2671

[49]

Lecouvez, M., Falgout, R.D., Woodward, C.S., Top, P.: A parallel multigrid reduction in time method for power systems. In: Power and Energy Society General Meeting (PESGM) pp. 1–5 (2016)

[50]

Lohner, R., Othmer, C., Mrosek, M., Figueroa, A., Degro, A.: Overnight industrial LES for external aerodynamics. In: AIAA Scitech 2020 Forum. American Institute of Aeronautics and Astronautics.

[51]

Margolin LG, Rider WJ, and Grinstein FF Modeling turbulent flow with implicit LES J. Turbul. 2006 7 1-27

[52]

Martin D, Colella P, Anghel M, and Alexander F Adaptive mesh refinement for multiscale nonequilibrium physics Comput. Sci. Eng. 2005 7 3 24-31

[53]

Martin DF, Colella P, and Graves D A cell-centered adaptive projection method for the incompressible Navier-Stokes equations in three dimensions J. Comput. Phys. 2008 227 3 1863-1886

[54]

McCorquodale P and Colella P A high-order finite-volume method for conservation laws on locally refined grids Commun. App. Math. Comput. Sci. 2011 6 1 1-25

[55]

Miniati F and Colella P Block structured adaptive mesh and time refinement for hybrid, hyperbolic + N-body systems J. Comput. Phys. 2007 227 1 400-430

[56]

Owen L, Guzik S, and Gao X A high-order adaptive algorithm for multispecies gaseous flows on mapped domains Comput. Fluids 2018 170 249-260

[57]

Owen LD, Gao X, and Guzik SM Techniques for improving monotonicity in a fourth-order finite-volume algorithm solving shocks and detonations J. Comput. Phys. 2020

[58]

Owen, L.D., Guzik, S.M., Gao, X.: High-order CFD modeling of multispecies flows. In: Fall Meeting of the Western States Section of the Combustion Institute (2015). WSSCI 2015-134IE-0030

[59]

Owen, L.D., Guzik, S.M., Gao, X.: A fourth-order finite-volume algorithm for compressible flow with chemical reactions on mapped grids (AIAA 2017-4498) (2017)

[60]

Sagaut, P.: Large Eddy Simulation for Incompressible Flows: An Introduction, 3 edn. Scientific Computation. Springer, Berlin (2006).

[61]

Schroder, J.B., Lecouvez, M., Falgout, R.D., Woodward, C.S., Top, P.: Parallel-in-time solution of power systems with scheduled events. Power and Energy Society General Meeting (PESGM) pp. 1–5 (2018)

[62]

Simon HD Parallel Computational Fluid Dynamics: Implementations and Results 1992 Cambridge MIT Press

[63]

Steinbach O Space-time finite element methods for parabolic problems Comput. Methods Appl. Math. 2015

[64]

Steinbach O and Yang H Comparison of algebraic multigrid methods for an adaptive space-time finite-element discretization of the heat equation in 3D and 4D: Comparison of AMG for adaptive space-time FEM in 3D and 4D Numer. Linear Algebra Appl. 2018 25 3 e2143

[65]

Sterck, H.D., Falgout, R.D., Friedhoff, S., Krzysik, O.A., MacLachlan, S.P.: Optimizing mgrit and parareal coarse-grid operators for linear advection. arXiv:1910.03726. (2019)

[66]

Van Straalen, B., Colella, P., Graves, D.T., Keen, N.: Petascale block-structured AMR applications without distributed meta-data. In: Proceedings, Part II. Lecture Notes in Computer Science. Euro-Par 2011 Parallel Processing—17th International Conference, Springer (2011). ISBN 978-3-642-23396-8

[67]

Yang Z Large-eddy simulation: past, present and the future Chin. J. Aeronaut. 2015 28 1 11-24

[68]

Yue, X., Shu, S., Xu, X., Bu, W., Pan, K.: Parallel-in-time with fully finite element multigrid for 2-d space-fractional diffusion equations. arXiv preprint arXiv:1805.06688 (2018)

[69]

Zhang Q, Johansen H, and Colella P A fourth-order accurate finite-volume method with structured adaptive mesh refinement for solving the advection-diffusion equation SIAM J. Sci. Comput. 2012 34 2 B179-B201

Cited By

Zhen MDing XQu KCai JPan S(2024)Enhancing the Convergence of the Multigrid-Reduction-in-Time Method for the Euler and Navier–Stokes EquationsJournal of Scientific Computing10.1007/s10915-024-02596-0100:2Online publication date: 21-Jun-2024
https://dl.acm.org/doi/10.1007/s10915-024-02596-0
Tanade CRakestraw ELadd WDraeger ERandles AMohror KArnold DBadia R(2023)Cloud Computing to Enable Wearable-Driven Longitudinal Hemodynamic MapsProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607101(1-14)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607101

Index Terms

A space-time parallel algorithm with adaptive mesh refinement for computational fluid dynamics

Index terms have been assigned to the content through auto-classification.

Recommendations

The Study of Swimming Propulsion Using Computational Fluid Dynamics
An unstructured adaptive mesh refinement approach for computational fluid dynamics of reacting flows
Abstract
A new Direct Numerical Simulation (DNS) code HAMISH with Adaptive Mesh Refinement (AMR) has been developed to simulate compressible reacting flow in a computationally economical manner. The focus is on problems where high gradients of ...
Highlights
- High-order Adaptive Mesh Refinement approach for turbulent combustion simulation.
Adaptive multiscale finite-volume method for nonlinear multiphase transport in heterogeneous formations

In the previous multiscale finite-volume (MSFV) method, an efficient and accurate multiscale approach was proposed to solve the elliptic flow equation. The reconstructed fine-scale velocity field was then used to solve the nonlinear hyperbolic transport ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computing and Visualization in Science

Computing and Visualization in Science Volume 23, Issue 1-4

Dec 2020

214 pages

ISSN:1432-9360

EISSN:1433-0369

Issue’s Table of Contents

© Springer-Verlag GmbH Germany, part of Springer Nature 2020.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 December 2020

Accepted: 31 July 2020

Received: 07 December 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhen MDing XQu KCai JPan S(2024)Enhancing the Convergence of the Multigrid-Reduction-in-Time Method for the Euler and Navier–Stokes EquationsJournal of Scientific Computing10.1007/s10915-024-02596-0100:2Online publication date: 21-Jun-2024
https://dl.acm.org/doi/10.1007/s10915-024-02596-0
Tanade CRakestraw ELadd WDraeger ERandles AMohror KArnold DBadia R(2023)Cloud Computing to Enable Wearable-Driven Longitudinal Hemodynamic MapsProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607101(1-14)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607101

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents