research-article

Cosmic microwave background map-making at the petascale and beyond

Authors:

Rajesh Sudarsan,

Julian Borrill,

Christopher Cantalupo,

Theodore Kisner,

Kamesh Madduri,

Horst SimonAuthors Info & Claims

ICS '11: Proceedings of the international conference on Supercomputing

Pages 305 - 316

https://doi.org/10.1145/1995896.1995944

Published: 31 May 2011 Publication History

Abstract

The analysis of Cosmic Microwave Background (CMB) observations is a long-standing computational challenge, driven by the exponential growth in the size of the data sets being gathered. Since this growth is projected to continue for at least the next decade, it will be critical to extend the analysis algorithms and their implementations to peta-scale high performance computing (HPC) systems and beyond. The most computationally intensive part of the analysis is generating and reducing Monte Carlo realizations of an experiment's data. In this work we take the current state-of-the-art simulation and mapping software and investigate its performance when pushed to tens of thousands of cores on a range of leading HPC systems, in particular focusing on the communication bottleneck that emerges at high concurrencies. We present a new communication strategy that removes this bottleneck, allowing for CMB analyses of unprecedented scale and hence fidelity. Experimental results show a communication speedup of up to 116x using our alternative strategy.

References

[1]

AMD Core Math Library, 2010. http://www.amd.com/acml.

[2]

P. Balaji, D. Buntinas, D. Goodell, W. Gropp, S. Kumar, E. Lusk, R. Thakur, and J.L. Träff. Mpi on a million processors. In Proc. 16th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface, pages 20--30. Springer-Verlag, 2009.

Digital Library

[3]

D. Baskins. Judy arrays web page, 2004. http://judy.sourceforge.net/.

[4]

J.R. Bond, A.H. Jaffe, and L. Knox. Estimating the power spectrum of the cosmic microwave background. Phys. Rev. D, 57(4):2117--2137, 1998.

[5]

COBE, 2010. http://lambda.gsfc.nasa.gov/product/cobe.

[6]

S. Dodelson. Modern Cosmology. Academic Press, 2003.

[7]

J. Dunkley et al. Five-year Wilkinson microwave anisotropy probe observations: Likelihoods and parameters from the wmap data. The Astrophysical Journal Supplement Series, 180(2):306, 2009.

[8]

C.M. Cantalupo et al. MADmap: A Massively Parallel Maximum Likelihood Cosmic Microwave Background Map-maker. The Astrophysical Journal Supplement Series, 187:212, 2010.

[9]

J. Pješivac-Grbović et al. Performance analysis of MPI collective operations. Cluster Computing, 10(2):127--143, 2007.

Digital Library

[10]

G.H. Golub and C.F. Van Loan. Matrix computations. Johns Hopkins University Press, 3rd edition, 1996.

[11]

E. Hivon et al. MASTER of the cosmic microwave background anisotropy power spectrum: A fast method for statistical analysis of large and complex cosmic microwave background data sets. The Astrophysical Journal, 567(1):2, 2002.

[12]

R.W. Hockney. The communication challenge for MPP: Intel Paragon and Meiko CS-2. Parallel Computing, 20(3):389--398, 1994.

Digital Library

[13]

T. Hoefler, C. Siebert, and A. Lumsdaine. Scalable communication protocols for dynamic sparse data exchange. In Proc. 15th ACM SIGPLAN symposium on Principles and practice of parallel programming (PPoPP '10), pages 159--168. ACM, 2010.

Digital Library

[14]

T. Hoefler and J.L. Träff. Sparse collective operations for mpi. In Proc. 14th Int'l. Workshop on High-Level Parallel Programming Models and Supportive Environments (HIPS09), 2009.

Digital Library

[15]

E. Komatsu et al. Five-year Wilkinson microwave anisotropy probe observations: Cosmological interpretation. The Astrophysical Journal Supplement Series, 180(2):330, 2009.

[16]

M3 data abstraction library, 2010. http://crd.lbl.gov/ cmc/M3/.

[17]

Intel Math Kernel Library, 2010. http://software.intel.com/en-us/intel-mkl.

[18]

The OpenMP API specifications, 2010. http://openmp.org/wp.

[19]

Planck science team home, 2010. http://www.rssd.esa.int/index.php?project=PLANCK.

[20]

R. Rabenseifner. New optimized MPI reduce algorithm, 2010. https://fs.hlrs.de/projects/par/mpi/myreduce.html.

[21]

R. Rabenseifner and J.L. Träff. More efficient reduction algorithms for non-power-of-two number of processors in message-passing parallel systems. In Proc. Recent Advances in Parallel Virtual Machine and Message Passing Interface, pages 309--335, 2004.

[22]

G.F. Smoot et al. Preliminary results from the COBE differential microwave radiometers - large angular scale isotropy of the cosmic microwave background. Astrophysics Journal, 371:L1-L5, 1991.

[23]

R. Stompor et al. Making maps of the cosmic microwave background: The MAXIMA example. Phys. Rev. D, 65(2):022003, 2001.

[24]

M. Tegmark. CMB mapping experiments: A designer's guide. Phys. Rev. D, 56(8):4514--4529, 1997.

[25]

R. Thakur, R. Rabenseifner, and W. Gropp. Optimization of collective communication operations in MPICH. Int'l. Journal of High Performance Computing Applications, 19(1):49--66, 2005.

Digital Library

[26]

Top500 supercomputer sites, 2010. http://top500.org.

[27]

WMAP, 2010. http://map.gsfc.nasa.gov.

Cited By

Saillard ESen KLavrijsen WIancu C(2018)Maximizing Communication Overlap with Dynamic Program AnalysisProceedings of the International Conference on High Performance Computing in Asia-Pacific Region10.1145/3149457.3149459(1-11)Online publication date: 28-Jan-2018
https://dl.acm.org/doi/10.1145/3149457.3149459
Grigori LStompor RSzydlarski MHollingsworth J(2012)A parallel two-level preconditioner for cosmic microwave background map-makingProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/2388996.2389120(1-10)Online publication date: 10-Nov-2012
https://dl.acm.org/doi/10.5555/2388996.2389120
Grigori LStompor RSzydlarski M(2012)A parallel two-level preconditioner for Cosmic Microwave Background map-making2012 International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC.2012.10(1-10)Online publication date: Nov-2012
https://doi.org/10.1109/SC.2012.10

Index Terms

Cosmic microwave background map-making at the petascale and beyond
1. Computing methodologies
  1. Parallel computing methodologies
    1. Parallel programming languages
2. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language types
        Parallel programming languages

Recommendations

Cosmic microwave background recovery: a graph-based Bayesian convolutional network approach
AAAI'23/IAAI'23/EAAI'23: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence

The cosmic microwave background (CMB) is a significant source of knowledge about the origin and evolution of our universe. However, observations of the CMB are contaminated by foreground emissions, obscuring the CMB signal and reducing its efficacy in ...
Progress towards petascale applications in biology: status in 2006
Euro-Par'06: Proceedings of the CoreGRID 2006, UNICORE Summit 2006, Petascale Computational Biology and Bioinformatics conference on Parallel processing

Petascale computing is currently a common topic of discussion in the high performance computing community. Biological applications, particularly protein folding, are often given as examples of the need for petascale computing. There are at present ...
Big Bang, Big Data, Big Iron: Fifteen Years of Cosmic Microwave Background Data Analysis at NERSC
The cosmic microwave background (CMB) consists of photons created in the Big Bang and cooled by the subsequent expansion of the universe to an almost perfectly uniform 3-Kelvin sky signal today. These photons experience the entire history of the universe, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICS '11: Proceedings of the international conference on Supercomputing

May 2011

398 pages

ISBN:9781450301022

DOI:10.1145/1995896

General Chair:
David K. Lowenthal
University of Arizona
,
Program Chairs:
Bronis R. de Supinski
Lawrence Livermore National Laboratory
,
Sally A. McKee
Chalmers University of Technology

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICS '11

Sponsor:

SIGARCH

ICS '11: International Conference on Supercomputing

May 31 - June 4, 2011

Arizona, Tucson, USA

Acceptance Rates

Overall Acceptance Rate 629 of 2,180 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
131
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Saillard ESen KLavrijsen WIancu C(2018)Maximizing Communication Overlap with Dynamic Program AnalysisProceedings of the International Conference on High Performance Computing in Asia-Pacific Region10.1145/3149457.3149459(1-11)Online publication date: 28-Jan-2018
https://dl.acm.org/doi/10.1145/3149457.3149459
Grigori LStompor RSzydlarski MHollingsworth J(2012)A parallel two-level preconditioner for cosmic microwave background map-makingProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/2388996.2389120(1-10)Online publication date: 10-Nov-2012
https://dl.acm.org/doi/10.5555/2388996.2389120
Grigori LStompor RSzydlarski M(2012)A parallel two-level preconditioner for Cosmic Microwave Background map-making2012 International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC.2012.10(1-10)Online publication date: Nov-2012
https://doi.org/10.1109/SC.2012.10

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten