Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Solving Parametric Partial Differential Equations with Deep Rectified Quadratic Unit Neural Networks

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Implementing deep neural networks for learning the solution maps of parametric partial differential equations (PDEs) turns out to be more efficient than using many conventional numerical methods. However, limited theoretical analyses have been conducted on this approach. In this study, we investigate the expressive power of deep rectified quadratic unit (ReQU) neural networks for approximating the solution maps of parametric PDEs. The proposed approach is motivated by the recent important work of Kutyniok et al. (Constr Approx 1–53, 2021), which uses deep rectified linear unit (ReLU) neural networks for solving parametric PDEs. In contrast to the previously established complexity-bound \(\mathcal {O}\left( d^3\log _{2}^{q}(1/ \epsilon ) \right) \) for ReLU neural networks, we derive an upper bound \(\mathcal {O}\left( d^3\log _{2}^{q}\log _{2}(1/ \epsilon ) \right) \) on the size of the deep ReQU neural network required to achieve accuracy \(\epsilon >0\), where d is the dimension of reduced basis representing the solutions. Our method takes full advantage of the inherent low-dimensionality of the solution manifolds and better approximation performance of deep ReQU neural networks. Numerical experiments are performed to verify our theoretical result.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

The dataset is available at www.github.com/MoGeist/diffusion_PPDE.

References

  1. Alnæs, M., Blechta, J., Hake, J., Johansson, A., Kehlet, B., Logg, A., Richardson, C., Ring, J., Rognes, M.E., Wells, G.N.: The fenics project version 1.5. Arch. Numer. Softw. 3(100), 9–23 (2015)

  2. Bachmayr, M., Cohen, A.: Kolmogorov widths and low-rank approximations of parametric elliptic PDEs. Math. Comput. 86(304), 701–724 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bhattacharya, K., Hosseini, B., Kovachki, N.B., Stuart, A.M.: Model reduction and neural networks for parametric PDEs. SMAI J. Comput. Math. 7, 121–157 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  4. Chui, C.K., Mhaskar, H.N.: Deep nets for local manifold learning. Front. Appl. Math. Stat. 4, 12 (2018)

    Article  Google Scholar 

  5. Dal Santo, N., Deparis, S., Pegolotti, L.: Data driven approximation of parametrized PDEs by reduced basis and neural networks. J. Comput. Phys. 416, 109550 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  6. Duan, C., Jiao, Y., Lai, Y., Li, D., Yang, J.Z., et al.: Convergence rate analysis for deep ritz method. Commun. Comput. Phys. 31(4), 1020–1048 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  7. Elbrachter, D., Grohs, P., Jentzen, A., Schwab, C.: Dnn expression rate analysis of high-dimensional pdes, Application to option pricing. Constr. Approx. 55(1), 3–71 (2018)

  8. Evans, L.C.: Partial Differential Equations, volume 19 of Graduate Studies in Mathematics, 2nd edn. American Mathematical Society, Providence, RI (2010)

    Google Scholar 

  9. Geist, M., Petersen, P., Raslan, M., Schneider, R., Kutyniok, G.: Numerical solution of the parametric diffusion equation by deep neural networks. J. Sci. Comput. 88(1), 1–37 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  10. Han, J., Jentzen, A., Weinan, E.: Solving high-dimensional partial differential equations using deep learning. Proc. Nat. Acad. Sci. 115(34), 8505–8510 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  11. He, J., Li, L., Xu, J., Zheng, C.: Relu deep neural networks and linear finite elements. J. Comput. Math. 38(3), 502 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  12. Hesthaven, J.S., Rozza, G., Stamm, B., et al.: Certified Reduced Basis Methods for Parametrized Partial Differential Equations, vol. 590. Springer, Berlin (2016)

    Book  MATH  Google Scholar 

  13. Khoo, Y., Lu, J., Ying, L.: Solving parametric PDE problems with artificial neural networks. Eur. J. Appl. Math. 32(3), 421–435 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  14. Kovachki, N., Lanthaler, S., Mishra, S.: On universal approximation and error bounds for fourier neural operators. J. Mach. Learn. Res., 22:Art–No, (2021)

  15. Kutyniok, G., Petersen, P., Raslan, M., Schneider, R.: A theoretical analysis of deep neural networks and parametric pdes. Constr. Approx. 55(1), 73–125 (2021)

  16. Lanthaler, S., Mishra, S., Karniadakis, G.: Error estimates for deeponets: a deep learning framework in infinite dimensions. IMA. 6, 1–141 (2022)

  17. Li, B., Tang, S., Haijun, Y.: Better approximations of high dimensional smooth functions by deep neural networks with rectified power units. Commun. Comput. Phys. 27, 379–411 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  18. Li, Z., Kovachki, N.B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., Anandkumar, A. et al.: Fourier neural operator for parametric partial differential equations. In: International Conference on Learning Representations (2020)

  19. Lu L., Pengzhan J., George E.K.: Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193, 2019

  20. Mhaskar, H.N.: Neural networks for optimal approximation of smooth and analytic functions. Neural Comput. 8(1), 164–177 (1996)

    Article  Google Scholar 

  21. Mhaskar, H.N.: Approximation properties of a multilayered feedforward artificial neural network. Adv. Comput. Math. 1(1), 61–80 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  22. Opschoor, J.A.A., Schwab, C., Zech, J.: Exponential relu DNN expression of holomorphic maps in high dimension. Constr. Approx. 55(1), 537–582 (2021)

  23. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: an imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32, 8026–8037 (2019)

    Google Scholar 

  24. Quarteroni, A., Manzoni, A., Negri, F.: Reduced Basis Methods for Partial Differential Equations: An Introduction, vol. 92. Springer, Berlin (2015)

    MATH  Google Scholar 

  25. Salvador, M., Dede, L., Manzoni, A.: Non intrusive reduced order modeling of parametrized PDEs by kernel pod and neural networks. Comput. Math. Appl. 104, 1–13 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  26. Schwab, Christoph, Zech, Jakob: Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ. Anal. Appl. (Singap.) 17(01), 19–55 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  27. Sirignano, J., Spiliopoulos, K.: Dgm: a deep learning algorithm for solving partial differential equations. J. Comput. Phys. 375, 1339–1364 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  28. Tang, S., Li, B., Yu, H.: Chebnet: efficient and stable constructions of deep neural networks with rectified power units using chebyshev approximations. arXiv preprint arXiv:1911.05467, (2019)

  29. Yarotsky, D.: Error bounds for approximations with deep ReLU networks. Neural Netw. 94, 103–114 (2017)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

We would like to thank the two anonymous reviewers for their helpful and constructive comments on our work.

Funding

The authors were in part supported by NSFC (Grant No. 11725102), National Support Program for Young Top-Notch Talents, and Shanghai Science and Technology Program (Project No. 21JC1400600, No. 19JC1420101, and No. 20JC1412700).

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, LMNS and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai, 200433, People’s Republic of China

    Zhen Lei, Lei Shi & Chenyu Zeng

Authors
  1. Zhen Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chenyu Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chenyu Zeng.

Ethics declarations

Competing interests

The authors have not disclosed any competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, Z., Shi, L. & Zeng, C. Solving Parametric Partial Differential Equations with Deep Rectified Quadratic Unit Neural Networks. J Sci Comput 93, 80 (2022). https://doi.org/10.1007/s10915-022-02015-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-022-02015-2

Keywords

Search

Navigation