Abstract
The BlueGene/L supercomputer will consist of 65,536 dual-processor compute nodes interconnected by two high-speed networks: a three-dimensional torus network and a tree topology network. Each compute node can only address its own local memory, making message passing the natural programming model for BlueGene/L. In this paper we present our implementation of MPI for BlueGene/L. In particular, we discuss how we leveraged the architectural features of BlueGene/L to arrive at an efficient implementation of MPI in this machine. We validate our approach by comparing MPI performance against the hardware limits and also the relative performance of the different modes of operation of BlueGene/L. We show that dedicating one of the processors of a node to communication functions greatly improves the bandwidth achieved by MPI operation, whereas running two MPI tasks per compute node can have a positive impact on application performance.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
The MPICH and MPICH2 homepage, http://www-unix.mcs.anl.gov/mpi/mpich
Adiga, N.R., et al.: An overview of the BlueGene/L supercomputer. In: SC 2002 – High Performance Networking and Computing, Baltimore, MD (November 2002)
Almási, G., Archer, C., Nos, J.G.C., Gupta, M., Martorell, X., Moreira, J.E., Gropp, W., Rus, S., Toonen, B.: MPI on BlueGene/L: Designing an Efficient General Purpose Messaging Solution for a Large Cellular System, September 2003. LNCS. Springer, Heidelberg (2003)
Almási, G., Bellofatto, R., Brunheroto, J., Casçaval, C., Nos, J.G.C., Ceze, L., Crumley, P., Erway, C., Gagliano, J., Lieber, D., Martorell, X., Moreira, J.E., Sanomiya, A., Strauss, K.: An overview of the BlueGene/L system software organization. In: Kosch, H., Böszörményi, L., Hellwagner, H. (eds.) Euro-Par 2003. LNCS, vol. 2790, pp. 543–555. Springer, Heidelberg (2003)
Almasi, G., et al.: Cellular supercomputing with system-on-a-chip. In: IEEE International Solid-state Circuits Conference ISSCC (2001)
Bailey, D., Harris, T., Saphir, W., van der Wijngaart, R., Woo, A., Yarrow, M.: The NAS Parallel Benchmarks 2.0. Technical Report NAS-95-020, NASA Ames Research Center (December 1995)
Chiola, G., Ciaccio, G.: Gamma: a low cost network of workstations based on active messages. In: Proc. Euromicro PDP 1997, London, UK, January 1997, IEEE Computer Society, Los Alamitos (1997)
Gropp, W., Lusk, E., Ashton, D., Ross, R., Thakur, R., Toonen, B.: MPICH Abstract Device Interface Version 3.4 Reference Manual: Draft of May 20 (2003), http://www-unix.mcs.anl.gov/mpi/mpich/adi3/adi3man.pdf
Pakin, S., Lauria, M., Chien, A.: High performance messaging on workstations: Illinois Fast Messages (FM) for Myrinet. In: Supercomputing 1995, San Diego, CA (December 1995)
Snir, M., Otto, S., Huss-Lederman, S., Walker, D., Dongarra, J.: MPI - The Complete Reference, 2nd edn. The MIT Press, Cambridge (2000)
Turner, D., Oline, A., Chen, X., Benjegerdes, T.: Integrating new capabilities into NetPIPE, September 2003. LNCS. Springer, Heidelberg (2003)
von Eicken, T., Basu, A., Buch, V., Vogels, W.: U-net: A user-level network interface for parallel and distributed computing. In: Proceedings of the 15th ACM Symposium on Operating Systems Principles, Copper Mountain, Colorado (December 1995)
von Eicken, T., Culler, D.E., Goldstein, S.C., Schauser, K.E.: Active Messages: a mechanism for integrated communication and computation. In: Proceedings of the 19th International Symposium on Computer Architecture (May 1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Almási, G. et al. (2004). Implementing MPI on the BlueGene/L Supercomputer. In: Danelutto, M., Vanneschi, M., Laforenza, D. (eds) Euro-Par 2004 Parallel Processing. Euro-Par 2004. Lecture Notes in Computer Science, vol 3149. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27866-5_112
Download citation
DOI: https://doi.org/10.1007/978-3-540-27866-5_112
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22924-7
Online ISBN: 978-3-540-27866-5
eBook Packages: Springer Book Archive