Abstract
This paper presents strong scaling performance data for the Blue Matter molecular dynamics framework using a novel n-body spatial decomposition and a collective communications technique implemented on both MPI and low level hardware interfaces. Using Blue Matter on Blue Gene/L, we have measured scalability through 16,384 nodes with measured time per time-step of under 2.3 milliseconds for a 43,222 atom protein/lipid system. This is equivalent to a simulation rate of over 76 nanoseconds per day and represents an unprecedented time-to-solution for biomolecular simulation as well as continued speed-up to fewer than three atoms per node. On a smaller, solvated lipid system with 13,758 atoms, we have achieved continued speedups through fewer than one atom per node and less than 2 milliseconds/time-step. On a 92,224 atom system, we have achieved floating point performance of over 1.8 TeraFlops/second on 16,384 nodes. Strong scaling of fixed-size classical molecular dynamics of biological systems to large numbers of nodes is necessary to extend the simulation time to the scale required to make contact with experimental data and derive biologically relevant insights.
Chapter PDF
Similar content being viewed by others
Keywords
- Classical Molecular Dynamic
- Strong Scaling
- Biomolecular Simulation
- Range Electrostatic Interaction
- Float Point Performance
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Fitch, B., Germain, R., Mendell, M., Pitera, J., Pitman, M., Rayshubskiy, A., Sham, Y., Suits, F., Swope, W., Ward, T., Zhestkov, Y., Zhou, R.: Blue Matter, an application framework for molecular simulation on Blue Gene. Journal of Parallel and Distributed Computing 63, 759–773 (2003)
Germain, R., Zhestkov, Y., Eleftheriou, M., Rayshubskiy, A., Suits, F., Ward, T., Fitch, B.: Early performance data on the Blue Matter molecular simulation framework. IBM Journal of Research and Development 49(2/3), 447–456 (2005)
Germain, R.S., Fitch, B., Rayshubskiy, A., Eleftheriou, M., Pitman, M.C., Suits, F., Giampapa, M., Ward, T.C.: Blue Matter on Blue Gene/L: massively parallel computation for biomolecular simulation. In: CODES+ISSS 2005: Proceedings of the 3rd IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis, pp. 207–212. ACM Press, New York (2005)
Allen, F., et al.: Blue Gene: a vision for protein science using a petaflop supercomputer. IBM Systems Journal 40(2), 310–327 (2001)
Pitman, M.C., Grossfield, A., Suits, F., Feller, S.E.: Role of cholesterol and polyunsaturated chains in lipid-protein interactions: Molecular dynamics simulation of rhodopsin in a realistic membrane environment. Journal of the American Chemical Society 127(13), 4576–4577 (2005)
Gara, A., et al.: Overview of the Blue Gene/L system architecture. IBM Journal of Research and Development 49(2/3), 195–212 (2005)
Adiga, N., et al.: Blue Gene/L torus interconnection network. IBM Journal of Research and Development 49(2/3), 265–276 (2005)
Bader, J., Chandler, D.: Computer simulation study of the mean forces between ferrous and ferric ions in water. The Journal of Physical Chemistry 96(15) (1992)
De Leeuw, S., Perram, J., Smith, E.: Simulation of electrostatic systems in periodic boundary conditions I. lattice sums and dielectric constants. Proc. Roy. Soc. Lond. A 373, 27–56 and references therein (1980)
Deserno, M., Holm, C.: How to mesh up ewald sums. J. Chem. Phys. 109(18), 7678–7693 (1998)
Plimpton, S., Hendrickson, B.: A new parallel method for molecular dynamics simulation of macromolecular systems. Journal of Computational Chemistry 17(3), 326–337 (1996)
Kale, L., Skeel, R., Bhandarkar, M., Brunner, R., Gursoy, A., Krawetz, N., Phillips, J., Shinozaki, A., Varadarajan, K., Schulten, K.: NAMD2: Greater scalability for parallel molecular dynamics. Journal of Computational Physics 151, 283–312 (1999)
Phillips, J., Zheng, G., Kumar, S., Kale, L.: NAMD: biomolecular simulation on thousands of processors. In: Supercomputing 2002 Proceedings (2002), http://www.sc2002.org/paperpdfs/pap.pap277.pdf
Kumar, S., Huang, C., Almasi, G., Kale, L.V.: Achieving strong scaling with NAMD on Blue Gene/l. In: 20th IEEE International Parallel and Distributed Processing Symposium. IEEE, Los Alamitos (2006), http://charm.cs.uiuc.edu/papers/NAMDIDPDS06.pdf
Snir, M.: A note on n-body computations with cutoffs. Theory of Computing Systems 37, 295–318 (2004), doi:10.1007/s00224-003-1071-0.
Shaw, D.E.: A fast, scalable method for the parallel evaluation of distance-limited pairwise particle interactions. Journal of Computational Chemistry 26(13), 1318–1328 (2005)
Kale, L., Kumar, S., Varadarajan, K.: A framework for collective personalized communication. In: Parallel and Distributed Processing Symposium, 2003. Proceedings International. IEEE, Los Alamitos (2003), http://dx.doi.org/10.1109/IPDPS.2003.1213166
Swope, W., Andersen, H., Berens, P., Wilson, K.: A computer simulation method for the calculation of equilibrium constants for the formation of physical clusters of molecules: Application to small water clusters. Journal of Chemical Physics 76, 637–649 (1982)
Tuckerman, M., Berne, B., Martyna, G.: Reversible multiple time scale molecular dynamics. J. Chem. Phys. 97(3), 1990–2001 (1992)
Almasi, G., et al.: Design and implementation of message-passing services for the Blue Gene/L supercomputer. IBM Journal of Research and Development 49(2/3), 393–406 (2005)
Giampapa, M., et al.: Blue Gene/L advanced diagnostics environment. IBM Journal of Research and Development 49(2/3), 319–332 (2005)
Eleftheriou, M., Fitch, B., Rayshubskiy, A., Ward, T., Germain, R.: Performance measurements of the 3d FFT on the Blue Gene/L supercomputer. In: Cunha, J.C., Medeiros, P.D. (eds.) Euro-Par 2005. LNCS, vol. 3648, pp. 795–803. Springer, Heidelberg (2005)
Suits, F., Pitman, M.C., Feller, S.E.: Molecular dynamics investigation of the structural properties of phosphatidylethanolamine lipid bilayers. Journal of Chemical Physics 122(24) (2005)
Pitman, M.C., Suits, F., Gawrisch, K., Feller, S.E.: Molecular dynamics investigation of dynamical properties of phosphatidylethanolamine lipid bilayers. Journal of Chemical Physics 122(24) (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fitch, B.G. et al. (2006). Blue Matter: Strong Scaling of Molecular Dynamics on Blue Gene/L. In: Alexandrov, V.N., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds) Computational Science – ICCS 2006. ICCS 2006. Lecture Notes in Computer Science, vol 3992. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11758525_113
Download citation
DOI: https://doi.org/10.1007/11758525_113
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34381-3
Online ISBN: 978-3-540-34382-0
eBook Packages: Computer ScienceComputer Science (R0)