Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Optimizing Parallel S n Sweeps on Unstructured Grids for Multi-Core Clusters

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In particle transport simulations, radiation effects are often described by the discrete ordinates (S n ) form of Boltzmann equation. In each ordinate direction, the solution is computed by sweeping the radiation flux across the grid. Parallel S n sweep on an unstructured grid can be explicitly modeled as topological traversal through an equivalent directed acyclic graph (DAG), which is a data-driven algorithm. Its traditional design using MPI model results in irregular communication of massive short messages which cannot be efficiently handled by MPI runtime. Meanwhile, in high-end HPC cluster systems, multicore has become the standard processor configuration of a single node. The traditional data-driven algorithm of S n sweeps has not exploited potential advantages of multi-threading of multicore on shared memory. These advantages, however, as we shall demonstrate, could provide an elegant solution resolving problems in the previous MPI-only design. In this paper, we give a new design of data-driven parallel S n sweeps using hybrid MPI and Pthread programming, namely Sweep-H, to exploit hierarchical parallelism of processes and threads. With special multi-threading techniques and vertex schedule policy, Sweep-H gets more efficient communication and better load balance. We further present an analytical performance model for Sweep-H to reveal why and when it is advantageous over former MPI counterpart. On a 64-node multicore cluster system with 12 cores per node, 768 cores in total, Sweep-H achieves nearly linear scalability for moderate problem sizes, and better absolute performance than the previous MPI algorithm on more than 16 nodes (by up to two times speedup on 64 nodes).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Downar T, Siegel A, Unal C. Science based nuclear energy systems enabled by advanced modeling and simulation at the extreme scale. Report of Workshop on Nuclear Energy, May 2009, http://science.energy.gov/~/media/ascr/pdf/programdocuments/docs/Sc_nework_shop_report.pdf.

  2. Baker R S, Alcouffe R E. Parallel 3-D Sn performance for MPI on cray-T3D. In Proc. Joint Int. Conf. Math. Methods and Supercomputing for Nuclear Applicat., Oct. 1997, pp.377-393.

  3. Baker R S, Koch K R. An Sn algorithm for the massively parallel CM-200 computer. Nuclear Science and Engineering, 1998, 28: 312–320.

    Google Scholar 

  4. Valiant L G. A bridging model for parallel computation. Communications of the ACM, 1990, 33(8): 103–111.

    Article  Google Scholar 

  5. Plimpton S, Hendrickson B, Burns S et al. Parallel algorithms for radiation transport on unstructured grids. In Proc. ACM/IEEE Conf. Super Computing, Nov. 2000, Article No.25.

  6. Plimpton S, Hendrickson B, Burns S et al. Parallel S n sweeps on unstructured grids: Algorithms for prioritization, grid partitioning, and cycle detection. J. American Nuclear Science and Engineering, 2005, 150(3): 267–283.

    Google Scholar 

  7. Mo Z Y, Zhang A Q, Cao X L. Towards a parallel framework of grid-based numerical algorithms on DAGs. In Proc. the 20th IPDPS, Apr. 2006, p.310.

  8. Hewitt C, Bishop P, Steiger R. A universal modular actor formalism for artificial intelligence. In Proc. the 3rd IJCAI, Aug. 1973, pp.235-245.

  9. Schloegel K, Karypis G, Kumar V. Parallel static and dynamic multi-constraint graph partitioning. Concurrency and Computation: Practice and Experience, 2002, 14(3): 219–240.

    Article  MATH  Google Scholar 

  10. Karypis G, Kumar V. Multi-level graph partitioning schemes. In Proc. ICPP, Aug. 1995, pp.113-122.

  11. Hendrickson B, Leland R. A multilevel algorithm for partitioning graph. In Proc. ACM/IEEE Conf. Super Computing, Dec. 1995, Article No.28.

  12. Zhang A Q. Research on scalable parallel data driven algorithms and applications [Ph.D. Thesis]. China Academy of Engineering Physics, 2009.

  13. Pautz S D. An algorithm for parallel S n sweeps on unstructured meshes. Nuclear Science and Engineering, 2002, 140(2): 111–136.

    Google Scholar 

  14. Nowak P, Nemanic M K. Radiation transport calculations on unstructured grids using a spatially decomposed and threaded algorithm. In Proc. Int. Conf. Mathematics and Computation, Reactor Physics and Environmental Analysis in Nuclear Applications, Sept. 1999, pp.379-390.

  15. Gong C Y, Liu J, Chi L H, Huang H W, Fang J Y, Gong Z H. GPU accelerated simulations of 3D deterministic particle transport using discrete ordinates method. Journal of Computational Physics, 2011, 230(15): 6010–6022.

    Article  MATH  Google Scholar 

  16. Lubeck O, Lang M, Srinivasan R, Johnson G. Implementation and performance modeling of deterministic particle transport (Sweep3D) on the IBM Cell/BE. Scientific Programming, 2009, 17(1/2): 199–208.

    Google Scholar 

  17. Pennycook S J, Hammond S D, Mudalige G R, Wright S A, Jarvis S A. On the acceleration of wavefront applications using distributed many-core architectures. The Computer Journal, 2012, 55(2): 138–153.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China

    Jie Yan (Student Member, IEEE), Guang-Ming Tan (Member, CCF, ACM, IEEE) & Ning-Hui Sun (Fellow, CCF, Member, ACM, IEEE)

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Jie Yan (Student Member, IEEE)

Authors
  1. Jie Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guang-Ming Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ning-Hui Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Yan.

Additional information

This work is supported by the National Natural Science Foundation of China under Grant Nos. 60925009, 61003062, 61033009, 61272134, and the National Basic 973 Program of China under Grant Nos. 2011CB302502 and 2012CB316502.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(DOC 47 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yan, J., Tan, GM. & Sun, NH. Optimizing Parallel S n Sweeps on Unstructured Grids for Multi-Core Clusters. J. Comput. Sci. Technol. 28, 657–670 (2013). https://doi.org/10.1007/s11390-013-1366-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-013-1366-9

Keywords

Search

Navigation