research-article

Large-Scale Materials Modeling at Quantum Accuracy: Ab Initio Simulations of Quasicrystals and Interacting Extended Defects in Metallic Alloys

Authors:

Bikash Kanungo,

Vishal Subramanian,

Gourab Panigrahi,

Phani Motamarri,

Paul Zimmerman,

Vikram GaviniAuthors Info & Claims

SC '23: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Article No.: 1, Pages 1 - 12

https://doi.org/10.1145/3581784.3627037

Published: 11 November 2023 Publication History

Abstract

Ab initio electronic-structure has remained dichotomous between achievable accuracy and length-scale. Quantum many-body (QMB) methods realize quantum accuracy but fail to scale. Density functional theory (DFT) scales favorably but remains far from quantum accuracy. We present a framework that breaks this dichotomy by use of three interconnected modules: (i) invDFT: a methodological advance in inverse DFT linking QMB methods to DFT; (ii) MLXC: a machine-learned density functional trained with invDFT data, commensurate with quantum accuracy; (iii) DFT-FE-MLXC: an adaptive higher-order spectral finite-element (FE) based DFT implementation that integrates MLXC with efficient solver strategies and HPC innovations in FE-specific dense linear algebra, mixed-precision algorithms, and asynchronous compute-communication. We demonstrate a paradigm shift in DFT that not only provides an accuracy commensurate with QMB methods in ground-state energies, but also attains an unprecedented performance of 659.7 PFLOPS (43.1% peak FP64 performance) on 619,124 electrons using 8,000 GPU nodes of Frontier supercomputer.

Supplemental Material

MP4 File - SC23 video presentation for "Large-Scale Materials Modeling at Quantum Accuracy: Ab Initio Simulations of Quasicrystals and Interacting Extended Defects in Metallic Alloys"

SC23 video presentation for ACM Gordon Bell Finalist presentation for paper "Large-Scale Materials Modeling at Quantum Accuracy: Ab Initio Simulations of Quasicrystals and Interacting Extended Defects in Metallic Alloys" by Sambit Das, Bikash Kanungo, Vishal Subramanian, Gourab Panigrahi, Phani Motamarri, David Rogers, Paul Zimmerman and Vikram Gavini

Download
278.35 MB

References

[1]

W. Kohn, L. J. Sham, Self-consistent equations including exchange and correlation effects, Phys. Rev. 140(4A) (1965) A1133.

[2]

Becke, A. D. (2014). Perspective: Fifty years of density-functional theory in chemical physics. J. Chem. Phys., 140(18), 18A301.

[3]

B. Kanungo, P.M. Zimmerman, V. Gavini, Exact exchange-correlation potentials from ground-state electron densities. Nature communications 10(1) (2019), 4497.

[4]

B. Kanungo, J. Hatch, P. M. Zimmerman, & V. Gavini, Exact and model exchange-correlation potentials for open-shell systems. arXiv preprint arXiv:2305.15620 (2023).

[5]

P. Motamarri et al., DFT-FE-A massively parallel adaptive finite-element code for large-scale density functional theory calculations. Comp. Phys. Commun. 246 (2020), 106853.

[6]

S. Das et al., Fast, scalable and accurate finite-element based ab initio calculations using mixed precision computing: 46 PFLOPS simulation of a metallic dislocation system. In Proceedings of the 2019 International Conference for High Performance Computing, Networking, Storage and Analysis, Art. No. 2.

[7]

D. Shechtman et al., Metallic phase with long-range orientational order and no translational symmetry. Phys. Rev. Lett. 53(20) (1984), 1951.

[8]

K. Deguchi et al., Quantum critical state in a magnetic quasicrystal. Nature materials 11(12) (2012), 1013.

[9]

K. Kamiya et al., Discovery of superconductivity in quasicrystal. Nature communications 9(1) (2018), 154.

[10]

H. Takakura et al., Atomic structure of the binary icosahedral Yb-Cd quasicrystal. Nature materials 6(1) (2007), 58.

[11]

T. M. Pollock, Weight loss with magnesium alloys, Science 328(5981) (2010) 986.

[12]

Z. Wu et al., Mechanistic origin and prediction of enhanced ductility in magnesium alloys. Science 359(6374) (2018) 447.

[13]

Sherrill, C. D., & Schaefer III, H. F. (1999). The configuration interaction method: Advances in highly correlated approaches. In Advances in quantum chemistry (Vol. 34, pp. 143--269). Academic Press.

[14]

K.D. Vogiatzis et al., Pushing configuration-interaction to the limit: Towards massively parallel MCSCF calculations. J. Chem. Phys. 147(18) (2017), 184111.

[15]

A.E. Rask, P.M. Zimmerman, Toward full configuration interaction for transition-metal complexes. J. Phys. Chem. A 125(7) (2021), 1598.

[16]

Bartlett, R. J., & Musiał, M. (2007). Coupled-cluster theory in quantum chemistry. Reviews of Modern Physics, 79(1), 291.

[17]

P.R. Nagy, Approaching the basis set limit of CCSD (T) energies for large molecules with local natural orbital coupled-cluster methods. J. Chem. Theory Comput. 15(10) (2019), 5275.

[18]

W.M.C. Foulkes et al., Quantum Monte Carlo simulations of solids. Rev. Mod. Phys. 73(1) (2001), 33.

[19]

Towards QMCPACK Performance Portability. URL: https://ecpannualmeeting.com/assets/overview/posters/QMCPACK_Kent_ECP_Houston_2020.pdf

[20]

https://asc.llnl.gov/sites/asc/files/2020-09/coral2_qmcpack_vfeb13.pdf

[21]

G. Kresse, J. Furthmüller, Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set, Phys. Rev. B 54(16) (1996), 11169.

[22]

P. Giannozzi et al., Advanced capabilities for materials modelling with QUANTUM ESPRESSO, J. Phys. Condens. Matter. 29(46) (2017), 465901.

[23]

F. Gygi et al., Large-scale electronic structure calculations of high-Z metals on the BlueGene/L platform. In Proceedings of the 2006 ACM/IEEE conference on Supercomputing.

[24]

M. Valiev et al., NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations, Comput. Phys. Commun. 181(9) (2010), 1477.

[25]

J. Hutter et al., CP2K: atomistic simulations of condensed matter systems, Wiley Interdiscip. Rev.: Comput. Mol. Sci. 4(1) (2014) 15.

[26]

A. N. Ziogas et al., A data-centric approach to extreme-scale ab initio dissipative quantum transport simulations. In Proceedings of the 2019 International Conference for High Performance Computing, Networking, Storage and Analysis, Art. No. 1.

[27]

V. Blum et al., Ab initio molecular simulations with numeric atom-centered orbitals. Comp. Phys. Commun. 180 (2009), 2175.

[28]

J. R. Chelikowsky, N. Troullier, Y. Saad, Finite-difference-pseudopotential method: Electronic structure calculations without a basis, Phys. Rev. Lett. 72 (1994), 1240.

[29]

Q. Xu et al., SPARC: Simulation Package for Ab-initio Real-space Calculations. Software X 15 (2021), 100709.

[30]

M. Dogan, K.H. Liou, J.R. Chelikowsky, Solving the electronic structure problem for over 100,000 atoms in real-space, Physical Review Materials, 7(6) (2023), L063001.

[31]

Y. Hasegawa et al., First-principles calculations of electron states of a silicon nanowire with 100,000 atoms on the K computer, In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis.

[32]

J.E. Pask, P.A. Sterne, Finite element methods in ab initio electronic structure calculations, Modell. Simul. Mater. Sci. Eng. 13 (2005), R71.

[33]

S. Das, et al., DFT-FE 1.0: A massively parallel hybrid CPU-GPU density functional theory code using finite-element discretization. Comp. Phys. Commun. 280 (2022), 108473.

[34]

D.R. Bowler, T. Miyazaki, Calculations for millions of atoms with density functional theory: linear scaling shows its potential, J Phys. Condens. Matter 22(7) (2010), 074207.

[35]

L. Lin, et al. Accelerating atomic orbital-based electronic structure calculation via pole expansion and selected inversion, J Phys. Condens. Matter 25 (2013), 295501.

[36]

J.-L. Fattebert et al., Modeling dilute solutions using first-principles molecular dynamics: computing more than a million atoms with over a million cores. In Proceedings of 2016 International Conference for High Performance Computing, Networking, Storage and Analysis.

[37]

W. Hu et al., 2.5 Million-Atom Ab Initio Electronic-Structure Simulation of Complex Metallic Heterostructures with DGDFT. In Proceedings of 2022 International Conference for High Performance Computing, Networking, Storage and Analysis.

[38]

W. Hu, et al. Adaptively compressed exchange operator for large-scale hybrid density functional calculations with applications to the adsorption of water on silicene. J. Chem. Theory Comput. 13(3) (2017), 1188.

[39]

Y. Shi, A. Wasserman, Inverse Kohn-Sham density functional theory: progress and challenges. J. Phys. Chem. Lett. 12(22) (2021), 5308.

[40]

T. Heaton-Burgess, F.A. Bulat, W. Yang, Optimized effective potentials in finite basis sets. Phys. Rev. Lett. 98(25) (2007), 256401.

[41]

A.P. Gaiduk, I.G. Ryabinkin, V.N. Staroverov, Removal of basis-set artifacts in Kohn-Sham potentials recovered from electron densities. J. Chem. Theory Comput. 9(9) (2013), 3959.

[42]

L.A. Curtiss et al. Assessment of Gaussian-2 and density functional theories for the computation of enthalpies of formation. J. Chem. Phys. 106(3) (1997), 1063.

[43]

P. Motamarri et al., Higher-order adaptive finite-element methods for Kohn-Sham density functional theory, J. Comput. Phys. 253 (2013), 308.

Digital Library

[44]

Y. Zhou et al., Self-consistent-field calculations using Chebyshev-filtered subspace iteration, J. Comput. Phys. 219(1) (2006), 172.

Digital Library

Cited By

Yang SLiu JJin FLu Y(2024)Integration of artificial intelligence and big data in materials science: New paradigms and scientific discoveriesChinese Science Bulletin10.1360/TB-2024-040469:32(4730-4747)Online publication date: 5-Jul-2024
https://doi.org/10.1360/TB-2024-0404
Zheng ZChen JZhao YSong LQin XAn H(2024)DB-SpGEMM: A Massively Distributed Block-Sparse Matrix-Matrix Multiplication for Linear-Scaling DFT CalculationsProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673159(1156-1165)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3673038.3673159
Zampini SZerbinati UTurkyyiah GKeyes DHuebl ASilvano CRobinson TWoodtli M(2024)PETScML: Second-Order Solvers for Training Regression Problems in Scientific Machine LearningProceedings of the Platform for Advanced Scientific Computing Conference10.1145/3659914.3659931(1-12)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3659914.3659931
Show More Cited By

Index Terms

Large-Scale Materials Modeling at Quantum Accuracy: Ab Initio Simulations of Quasicrystals and Interacting Extended Defects in Metallic Alloys
1. Computing methodologies
  1. Modeling and simulation
    1. Simulation types and techniques
      1. Massively parallel and high-performance simulations
      2. Quantum mechanic simulation

Recommendations

Duality quantum computer and the efficient quantum simulations

Duality quantum computing is a new mode of a quantum computer to simulate a moving quantum computer passing through a multi-slit. It exploits the particle wave duality property for computing. A quantum computer with n qubits and a qudit simulates a ...
Large scale ab-initio simulations of dislocations
Highlights
- Sub-linear scaling density functional method for metals is developed.
- Ab-initio simulation of dislocation cores with more than 250,000 atoms.
- Petascale ab-initio simulations of dislocation in Magnesium with ∼70-90% efficiency up to ...
Abstract
We present a novel methodology to compute relaxed dislocations core configurations, and their energies in crystalline metallic materials using large-scale ab-intio simulations. The approach is based on MacroDFT, a coarse-grained density ...
The many-body Wigner Monte Carlo method for time-dependent ab-initio quantum simulations

The aim of ab-initio approaches is the simulation of many-body quantum systems from the first principles of quantum mechanics. These methods are traditionally based on the many-body Schrodinger equation which represents an incredible mathematical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '23: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

November 2023

1428 pages

ISBN:9798400701092

DOI:10.1145/3581784

Chair:
Dorian Arnold,
Program Chair:
Rosa M Badia,
Program Co-chair:
Kathryn Mohror

Copyright © 2023 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

U.S. Department of Energy, Office of Science, Basic Energy Sciences
Basic Energy Sciences
Toyota Research Institute
Department of Science and Technology, India
U.S. Air Force Office of Scientific Research
U.S. Department of Energy

Conference

SC '23

Sponsor:

SIGHPC

SC '23: International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 17, 2023

CO, Denver, USA

Acceptance Rates

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
3,513
Total Downloads

Downloads (Last 12 months)1,785
Downloads (Last 6 weeks)57

Reflects downloads up to 14 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang SLiu JJin FLu Y(2024)Integration of artificial intelligence and big data in materials science: New paradigms and scientific discoveriesChinese Science Bulletin10.1360/TB-2024-040469:32(4730-4747)Online publication date: 5-Jul-2024
https://doi.org/10.1360/TB-2024-0404
Zheng ZChen JZhao YSong LQin XAn H(2024)DB-SpGEMM: A Massively Distributed Block-Sparse Matrix-Matrix Multiplication for Linear-Scaling DFT CalculationsProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673159(1156-1165)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3673038.3673159
Zampini SZerbinati UTurkyyiah GKeyes DHuebl ASilvano CRobinson TWoodtli M(2024)PETScML: Second-Order Solvers for Training Regression Problems in Scientific Machine LearningProceedings of the Platform for Advanced Scientific Computing Conference10.1145/3659914.3659931(1-12)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3659914.3659931
Wu WZhou ZJiang QFeng JQin XMa HCao ZChen JChen SMeng XHou BXiong YWang LSun YAn HYang JHu W(2024)Enabling 13K-Atom Excited-State GW Calculations via Low-Rank Approximations and HPC on the New Sunway SupercomputerSC24: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41406.2024.00067(1-14)Online publication date: 17-Nov-2024
https://doi.org/10.1109/SC41406.2024.00067
You XXuan ZYang HLuan ZLiu YQian D(2024)GVARP: Detecting Performance Variance on Large-Scale Heterogeneous SystemsSC24: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41406.2024.00063(1-16)Online publication date: 17-Nov-2024
https://doi.org/10.1109/SC41406.2024.00063
Shang HLiu YWu ZChen ZLiu JShao MLi YKan BCui HFeng XZhang YTruhlar DAn HHe XYang J(2024)Pushing the Limit of Quantum Mechanical Simulation to the Raman Spectra of a Biological System with 100 Million AtomsSC24: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41406.2024.00011(1-12)Online publication date: 17-Nov-2024
https://doi.org/10.1109/SC41406.2024.00011
Razakh TLinker TLuo YKalia RNomura KVashishta PNakano A(2024)Accelerating Quantum Light-Matter Dynamics on Graphics Processing Units2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)10.1109/IPDPSW63119.2024.00176(1057-1066)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPSW63119.2024.00176
Babar MZhu ZKurchin RKaxiras EViswanathan V(2024)Twisto-Electrochemical Activity Volcanoes in Trilayer GrapheneJournal of the American Chemical Society10.1021/jacs.4c03464146:23(16105-16111)Online publication date: 3-Jun-2024
https://doi.org/10.1021/jacs.4c03464
Subramanian VDas SGavini V(2024)Tucker Tensor Approach for Accelerating Fock Exchange Computations in a Real-Space Finite-Element Discretization of Generalized Kohn–Sham Density Functional TheoryJournal of Chemical Theory and Computation10.1021/acs.jctc.4c0001920:9(3566-3579)Online publication date: 25-Apr-2024
https://doi.org/10.1021/acs.jctc.4c00019
Panigrahi GKodali NPanda DMotamarri P(2024)Fast hardware-aware matrix-free algorithms for higher-order finite-element discretized matrix multivector products on distributed systemsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2024.104925192:COnline publication date: 25-Sep-2024
https://dl.acm.org/doi/10.1016/j.jpdc.2024.104925

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten