Abstract
In this work we explore the use of deep learning models based on deep feedforward neural networks to solve ordinary and partial differential equations. The illustration of this methodology is given by solving a variety of initial and boundary value problems. The numerical results, obtained based on different feedforward neural networks structures, activation functions and minimization methods, were compared to each other and to the exact solutions. The neural network was implemented using the Python language, with the Tensorflow library.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Change history
23 January 2022
Chapter “Automated Housing Price Valuation and Spatial Data” was previously published non-open access. It has now been changed to open access under a CC BY 4.0 license and the copyright holder updated to ‘The Author(s)’.
Chapter “Solving Differential Equations Using Feedforward Neural Networks” was previously published open access. It now has been changed to non-open access and the copyright holder updated to “Springer Nature Switzerland AG”.
References
Abadi, M., et al.: TensorFlow: a system for large-scale machine learning. In: OSDI 2016, pp. 265–283. USENIX Association, USA (2016)
Bellman, R.: Dynamic Programming. Princeton University Press (1957)
Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5, 157–166 (1994). https://doi.org/10.1109/72.279181
Bottou, L., Curtis, E.F., Nocedal, J.: Optimization methods for large-scale machine learning. SIAM Rev. 60(2), 223–311 (2018). https://doi.org/10.1137/16M1080173
Chaudhari, P., Oberman, A., Osher, S., Soatto, S., Carlier, G.: Deep relaxation: partial differential equations for optimizing deep neural networks. Res. Math. Sci. 5(3), 1–30 (2018). https://doi.org/10.1007/s40687-018-0148-y
Curry, H.B.: The method of steepest descent for non-linear minimization problems. Quart. Appl. Math. 2, 258–261 (1944). https://doi.org/10.1090/qam/10667
Cybenko, G.: Approximation by superposition of a sigmoidal function. Math. Control Sig. Syst. 2, 303–314 (1989)
Dockhorn, T.: A discussion on solving partial differential equations using neural networks. arXiv:1904.07200 (2019)
Weinan, E., Han, J., Jentzen, A.: Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning. CoRR abs/2008.13333 (2020). https://arxiv.org/abs/2008.13333
Guo, Q., Liu, J.G., Wang, D.H.: A modified BFGS method and its superlinear convergence in nonconvex minimization with general line search rule. J. Appl. Math. Comput. 28, 435–446 (2008). https://doi.org/10.1007/s12190-008-0117-5
Guo, Y., Cao, X., Bainian, L., Gao, M.: Solving partial differential equations using deep learning and physical constraints. Appl. Sci. 10, 5917 (2020). https://doi.org/10.3390/app10175917
Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the Marquardt algorithm. IEEE Trans. Neural Netw. 5(6), 989–993 (1994). https://doi.org/10.1109/72.329697
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989). https://doi.org/10.1016/0893-6080(89)90020-8
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)
Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Netw. 9(5), 987–1000 (1998). https://doi.org/10.1109/72.712178
Lau, M.M., Hann Lim, K.: Review of adaptive activation function in deep neural network. In: 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES), pp. 686–690 (2018). https://doi.org/10.1109/IECBES.2018.8626714
Leake, C., Mortari, D.: Deep theory of functional connections: a new method for estimating the solutions of partial differential equations. Mach. Learn. Knowl. Extr. 2(1), 37–55 (2020). https://doi.org/10.3390/make2010004
Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989)
Parisi, D.R., Mariani, M.C., Laborde, M.A.: Solving differential equations with unsupervised neural networks. Chem. Eng. Process. Process Intensif. 42(8), 715–721 (2003). https://doi.org/10.1016/S0255-2701(02)00207-6
Raissi, M., Karniadakis, G.E.: Hidden physics models: machine learning of nonlinear partial differential equations. J. Comput. Phys. 357, 125–141 (2018). https://doi.org/10.1016/j.jcp.2017.11.039
Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22(3), 400–407 (1951). https://doi.org/10.1214/aoms/1177729586
Rumelhart, D.E., Willians, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986). https://doi.org/10.1038/323533a0
Samaniego, E., et al.: An energy approach to the solution of partial differential equations in computational mechanics via machine learning: concepts, implementation and applications. Comput. Meth. Appl. Mech. Eng. 362, 112790 (2020). https://doi.org/10.1016/j.cma.2019.112790
Sirignano, J., Spiliopoulos, K.: DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys. 375, 1339–1364 (2018). https://doi.org/10.1016/j.jcp.2018.08.029
Yu, Y., et al.: Dynamic control flow in large-scale machine learning. In: Proceedings of the 13th EuroSys Conference, EuroSys 2018, Association for Computing Machinery, New York (2018)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Guasti Junior, W., Santos, I.P. (2021). Solving Differential Equations Using Feedforward Neural Networks. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2021. ICCSA 2021. Lecture Notes in Computer Science(), vol 12952. Springer, Cham. https://doi.org/10.1007/978-3-030-86973-1_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-86973-1_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86972-4
Online ISBN: 978-3-030-86973-1
eBook Packages: Computer ScienceComputer Science (R0)