Abstract
Earthquakes, as one of the most disruptive natural hazards, have been a major research target for generations of scientists. Numerical simulation of earthquakes, as one of the few methods to verify and improve scientists’ understanding about the earthquake process, and a key tool in various earthquake engineering applications, has long been both an important and challenging application on supercomputers. In this paper, we discuss the major challenges for developing an accurate earthquake simulation tool on supercomputers. Based on the discussion, we then demonstrate our efforts on performing extreme-scale earthquake simulations on Sunway TaihuLight, a 125-Pflops machine with over 10 million heterogeneous cores. With systematic approaches to resolve the memory bandwidth constraint, we manage to achieve 8% to 16% efficiency for utilizing the entire machine to simulate Tangshan and Wenchuan Earthquakes with an unprecedented spatial resolution.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bao, H., Bielak, J., Ghattas, O., Kallivokas, L.F., O’hallaron, D.R., Shewchuk, J.R., Xu, J.: Earthquake ground motion modeling on parallel computers. In: Proceedings of the 1996 ACM/IEEE conference on Supercomputing, IEEE Computer Society, p. 13. (1996)
Cui, Y., Moore, R., Olsen, K., Chourasia, A., Maechling, P., Minster, B., Day, S., Hu, Y., Zhu, J., Majumdar, A., et al.: Enabling very-large scale earthquake simulations on parallel machines. In: International Conference on Computational Science, pp. 46–53. Springer (2007)
Cui, Y., Olsen, K.B., Jordan, T.H., Lee, K., Zhou, J., Small, P., Roten, D., Ely, G., Panda, D.K., Chourasia, A., et al.: Scalable earthquake simulation on petascale supercomputers. In: High Performance Computing, Networking, Storage and Analysis (SC), 2010 International Conference for IEEE, pp. 1–20. (2010)
Cui, Y., Poyraz, E., Olsen, K.B., Zhou, J., Withers, K., Callaghan, S., Larkin, J., Guest, C., Choi, D., Chourasia, A., et al.: Physics-based seismic hazard analysis on petascale heterogeneous supercomputers. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, ACM, p. 70. (2013)
Fu, H., He, C., Chen, B., Yin, Z., Zhang, Z., Zhang, W., Zhang, T., Xue, W., Liu, W., Yin, W., et al.: 18.9-pflops nonlinear earthquake simulation on Sunway Taihulight: Enabling depiction of 18-Hz and 8-m scenarios. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, ACM, p. 2. (2017)
Fu, H., Liao, J., Xue, W., Wang, L., Chen, D., Gu, L., Xu, J., Ding, N., Wang, X., He, C., et al.: Refactoring and optimizing the community atmosphere model (CAM) on the sunway taihulight supercomputer. In: High Performance Computing, Networking, Storage and Analysis, SC16: International Conference for IEEE, pp. 969–980.
Fu, H., Liao, J., Yang, J., Wang, L., Song, Z., Huang, X., Yang, C., Xue, W., Liu, F., Qiao, F., et al.: The Sunway Taihulight supercomputer: system and applications. Sci. China Inf. Sci. 59(7), 072001 (2016)
Harris, R.A., Barall, M., Aagaard, B., Ma, S., Roten, D., Olsen, K., Duan, B., Liu, D., Luo, B., Bai, K., et al.: A suite of exercises for verifying dynamic earthquake rupture codes. Seismol. Res. Lett. 89(3), 1146–1162 (2018)
Komatitsch, D., Tsuboi, S., Ji, C., Tromp, J.: A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the earth simulator. In: Supercomputing, 2003 ACM/IEEE Conference, IEEE, pp. 4–4. (2003)
Milne, J.: Earthquakes and other earth movements, vol. 56. D. Appleton and company, New York, USA (1886)
Nguyen, A., Satish, N., Chhugani, J., Kim, C., Dubey, P.: 3.5-d blocking optimization for stencil computations on modern cpus and gpus. In: High Performance Computing, Networking, Storage and Analysis (SC), 2010 International Conference for IEEE, pp. 1–13. (2010)
Olsen, K.B.: Simulation of three-dimensional wave propagation in the salt lake basin. Ph.D. thesis, Department of Geology and Geophysics, University of Utah (1994)
Roten, D., Cui, Y., Olsen, K.B., Day, S.M., Withers, K., Savran, W.H., Wang, P., Mu, D.: High-frequency nonlinear earthquake simulations on petascale heterogeneous supercomputers. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, p. 82. IEEE Press (2016)
Stein, S., Wysession, M.: An Introduction to Seismology, Earthquakes, and Earth Structure. Wiley, New York (2009)
Yang, C., Xue, W., Fu, H., You, H., Wang, X., Ao, Y., Liu, F., Gan, L., Xu, P., Wang, L., et al.: 10M-core scalable fully-implicit solver for nonhydrostatic atmospheric dynamics. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, IEEE Press, p. 6. (2016)
Zhang, Z., Zhang, W., Chen, X.: Three dimensional curved grid finite-difference method modelling for non-planar rupture dynamics. Geophys. J. Int. 199(2), 860–879 (2014). https://doi.org/10.1093/gji/ggu308
Zhang, Z., Zhang, W., Chen, X., Li, P., Fu, C.: Rupture dynamics and ground motion from potential earthquakes around Taiyuan. China Bull. Seismol. Soc. Am. 107(3), 1201–1212 (2017). https://doi.org/10.1785/0120160239
Acknowledgements
This work was supported in part by the National Key R&D Program of China (Grant No. 2017YFA0604500), by the National Natural Science Foundation of China (Grant No. 51761135015), and by Center for High Performance Computing and System Simulation, Pilot National Laboratory for Marine Science and Technology (Qingdao).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
This work was supported in part by the National Key R&D Program of China (Grant No. 2017YFA0604500), by the National Natural Science Foundation of China (Grant Nos. 91530323, 5171101179).
Rights and permissions
About this article
Cite this article
Fu, H., Chen, B., Zhang, W. et al. Extreme-scale earthquake simulations on Sunway TaihuLight. CCF Trans. HPC 1, 14–24 (2019). https://doi.org/10.1007/s42514-019-00004-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42514-019-00004-w